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Abstract.  This  paper  discusses  how  local  measurements  of  three-dimensional 
positions  and  surface  normals  (recorded  by  a  set  of  tactile  sensors,  or  by  three- 
dimensional  range  sensors),  may  be  used  to  identify  and  locate  objects,  from  among 
a  set  of  known  objects.  The  objects  arc  modeled  as  poiyhedra  having  up  to  six 
degrees  of  freedom  relative  to  the  sensors.  We  show  that  inconsistent  hypotheses 
about  pairings  between  sensed  points  and  object  surfaces  can  be  discarded  efficiently 
by  using  local  constraints  on;  distances  between  faces,  angles  between  face  normals, 
and  angles  (relative  to  the  surface  normals)  of  vectors  between  sensed  points.  We 
show  by  simulation  and  by  mathematical  bounds  that  the  number  of  hypotheses 
consistent  with  these  constraints  is  small.  We  also  show  how  to  recover  the  position 
and  orientation  of  the  object  from  the  sense  data.  The  algorithm's  performance  on 
data  obtained  from  a  triangulation  range  sensor  is  illustrated. 
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1.  The  Problem  and  the  Approach 


A  centra)  characteristic  of  advanced  applications  in  robotics  is  the  presence  of 
significant  uncertainty  about  the  identities  and  positions  of  obj<  ts  in  the  workspace 
of  the  robot.  It.  is  this  characteristic  that  makes  sensing  of  the  exU.nal  enviromneir 
an  essential  component  of  robot  systems.  The  process  of  sensing  can  be  loosely 
divided  into  two  stages:  first,  the  measurements  of  properties  of  the  objects  in  the 
environment,  and  second,  the  interpretation  of  those  measurements.  In  the  present 
paper,  we  will  concentrate  on  the  interpretation  of  sensory  data.  In  investigating 
this  problem,  we  make  only  a  few,  simple  assumptions  about  available  sensory 
measurements,  rather  than  considering  specific  details  of  a  particular  sensor.  As  a 
consequence,  the  interpretation  technique  that  is  developed  here  should  be  applicable 
to  a  wide  range  of  sensing  modalities.  As  well,  the  interpretation  technique  may 
have  implications  for  the  design  of  three-dimensional  sensors. 

1.1.  Problem  Definition 

The  specific  problem  we  consider  in  this  paper  is  to  identify  an  object  from 
among  a  set  of  known  objects  and  to  locate  it  relative  to  the  sensor.  The  object 
sensed  is  assumed  to  be  a  single,  possibly  non  convex,  polyhedral  object  (for  which 
we  have  an  accurate  geometric  model).  The  object  may  have  up  to  six  degrees 
of  freedom  relative  to  the  sensor  (three  translational  and  three  rotational).  The 
sensor,  which  could  be  tactile  or  range,  is  assumed  to  be  capable  of  providing 
three-dimensional  information  about  the  position  and  local  surface  orientation  of  a 
small  set  of  points  on  the  object.  Each  sensor  point  is  processed  to  obtain: 

1.  Surface  points  —  On  the  basis  of  sensor  readings,  the  positions  of  some 
points  on  the  sensed  object  can  be  determined  to  lie  within  some  small 
volume  relative  to  the  sensor, 

2.  Surface  normals  —  At  the  sensed  points,  the  surface  normal  of  the  object’s 
surface  can  be  recovered  to  within  some  cone  of  uncertainty. 

Our  goal  is  to  use  local  information  about  sensed  points  to  determine  the  set 
of  positions  and  orientations  of  an  object  that  are  consistent  with  the  sensed  data. 
If  there  are  no  consistent  positions  and  orientations,  the  object  is  excluded  from 
the  set  of  possible  objects. 

In  this  paper  we  do  not  discuss  how  surface  points  and  normals  may  be  obtained 
from  actual  sensor  data,  since  this  process  is  highly  sensor-dependent  (for  references 
to  existing  measurement  methods  see  Section  1.3).  Our  aim  is  to  show,  instead, 
how  such  data  may  be  used  in  conjunction  with  object  models  to  recognize  and 
localize  objects.  The  method,  in  turn,  suggests  criteria  for  the  design  of  sensors  and 
sensor-processing  strategies. 

Our  only  assumption  about  the  input  data  is  that  fairly  accurate  positions 
of  surface  points  are  obtainable  from  the  sensor,  but  that  significant  errors  exist 
in  determining  norma)  informatio/  This  assumption  reflects  the  type  of  data 
obtainable  from  tactile  sensors.  Range  sensors  based  on  triangulation  can  be  used 
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to  obtain  high  quality  measurements  of  normals  from  Matches  of  depth  data.  The 
availability  of  good  normal  data  merely  increases  t  he  clhciency  of  the  method. 

1.2.  Approach 

A  recent  paper  [Gaston  and  Eozano-Perez  83]  introduced  a  new  approach  to 
tactile  recognition  and  localization  for  polyhedra  with  three  degrees  of  positional 
freedom  (two  translational  and  one  rotational).  The  present  paper  generalizes  that 
approach  to  polyhedra  with  six  degrees  of  positional  freedom.  The  inputs  to  the 
recognition  process  arc:  a  set  of  sensed  points  and  normals,  and  a  set  of  geometric 
object  models  for  the  known  objects.  The  recognition  process,  as  outlined  in  the 
earlier  paper,  proceeds  in  two  steps: 

1 .  Generate  Feasible  Interpretations:  A  set  of  feasible  interpretations  of  the 
sense  data  is  constructed.  Interpretations  consist  of  pairings  of  each  sensed 
point  with  some  object  surface  of  one  of  the  known  objects.  Interpretations 
inconsistent  with  local  constraints  (derived  from  the  model)  on  the  sense 
data  are  discarded. 

2.  Model  Test:  The  feasible  interpretations  are  tested  for  consistency  with 
surface  equations  obtained  from  the  object  models.  An  interpretation  is 
legal  if  it  is  possible  to  solve  for  a  rotation  and  translation  that  would 
place  each  sense  point  on  an  object  surface.  The  sensed  point  must  lie 
inside  the  object  face,  not  just  on  the  surface. 

The  first  step  is  the  key  to  this  process.  The  number  of  possible  interpretations 
given  s  sensed  points  and  n  surfaces  is  n5.  Therefore,  it  is  not  feasible  to  carry  out 
a  model  test  on  all  possible  interpretations.  The  goal  of  the  recognition  algorithm  is 
to  exploit  the  local  constraints  on  the  sensed  data  so  as  to  minimize  the  number  of 
interpretations  that  need  testing.  This  approach  is  an  instance  of  a  classic  paradigm 
of  artificial  intelligence:  generate  and  test;  see  for  example  [Buchanan,  et  al.  69]. 

Consider  a  simple  example  of  the  approach,  illustrated  in  Figure  1.  The  model 
is  a  right  triangle,  with  edge  sizes  of  3,  4,  and  5  respectively.  From  this  model,  we 
can  construct  a  table  of  ranges  of  distances  between  pairs  of  points  on  the  edges. 
The  table  is  as  follows: 


Distance  Kang 

es  Between  Edges 

t 

2 

3 

1 

|0,3] 

_ _  [ 

M 

2 

_  10,51 _ 

..  [0,41  . 

[0,3] 

3 

_ [ML _ 1 

_ [0,5] 

Now,  suppose  we  know  the  positions  of  the  three  sensed  point  s,  }\  through  P3,  shown 
in  Figure  1.  The  measured  distances  between  those  points  are  di .  =  3-5, 

dist(P\,  P:\)  —  4.4,  aist{P2,  P3)  =  0.8.  From  this  we  see  that  any  interpretation  of 
the  sensed  points  that  assigns  P\  and  P2  both  to  edge  1  is  inconsistent  with  the 
model.  Similarly,  assigning  Pi  arid  / 2  to  edges  2  and  3  is  not  consistent.  Many  other 
pairwise  assignments  of  points  to  edges  can  be  discarded  simply  by  comparing  the 
measured  distances  to  the  ranges  in  the  table.  Note  that,  the  sensed  positions  are 
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Figure  i.  An  example  of  the  approach 


subject  to  error,  so  that  a  range  of  actual  distances  is  consistent  with  the  measured 
positions.  It  is  these  distance  ranges  that  must  be  compared  against  the  ranges 
in  the  table.  For  this  example,  only  6  of  the  27  possible  assignments  of  the  three 
points  to  the  three  model  edges  are  legal. 

Of  the  six  interpretations  consistent  with  the  distance  ranges,  the  two  shown 
in  Figure  1,  are  completely  consistent  once  the  line  equations  of  the  edges  are  taken 
into  account.  Each  of  these  interpretations  leads  to  a  solution  for  the  position  and 
orientation  of  the  triangle  relative  to  the  sensor.  Furthermore,  these  positions  and 
orientations  of  the  triangle  place  the  measured  points  inside  the  finite  edges,  not 
just  on  the  infinite  line. 

This  paper  discusses  both  steps  of  the  recognition  process,  focusing  first 
on  the  generate  step  and  then  considering  the  model  testing  stage.  We  show,  by 
mathematical  analysis  and  by  simulation,  that  the  number  of  feasible  interpretations 
can  be  reduced  to  manageable  numbers  by  the  use  of  local  geometric  constraints. 
In  particular,  we  investigate  the  effectiveness  of  the  different  local  constraints  and 
the  impact  of  measurement  errors  on  their  effectiveness.  We  further  show  that  the 
few  remaining  feasible  interpretations  can  efficiently  be  subjected  to  an  explicit 
model  test,  generally  resulting  in  a  single  interpretation  of  the  sense  data  (up  to 
symmetries).  We  also  illustrate  the  performance  of  the  algorithm  on  range  data 
obtained  by  triangulation. 
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1.3.  Three  Dimensional  Sensing 

Sensors  can  be  roughly  divided  into  two  categories:  non  contact  and  contact. 
Non  contact  sensing,  especially  visual  sensing,  has  received  extensive  attention  in 
the  robotics  and  artificial  intelligence  literature.  Contact  sensing,  such  as  tactile  or 
haptic  sensing,  plays  an  equally  important  role  in  robotics,  but  has  received  much 
less  attention.  In  this  paper,  our  aim  is  to  develop  a  sensory  interpretat  ion  method 
that  is  applicable  to  data  from  both  contact  and  non  contact  sensors. 

While  two-dimensional  sensing,  for  example  silhouette  or  binary  vision,  may 
be  adequate  for  restricted  situations  such  as  problems  with  three  degrees  of  freedom 
in  positioning,  the  general  localization  and  recognition  problem  requires  three- 
dimensional  sensing.  Throughout  this  paper,  we  will  concent  rate  or;  the  six-degree 
of  freedom  recognition  and  localization  problem  and  the  use  of  three-dimensional 
sensing.  Restrictions  of  the  method  to  the  simpler  case  of  three  degrees  of  freedom 
arc  straightforward. 

1.3.1.  Previous  Work  in  V  isual  Range  Sensing 

The  measurement  stage  of  visual  sensing  has  received  extensive  attention  in  the 
literature.  Of  particular  interest  here  are  methods  for  obtaining  three-dimensional 
position  and  surface  normal  information;  see  [Jarvis  83j  for  a  detailed  survey. 
Possible  methods  include  edge-based  stereo  systems  ;e.g.  Crimson  SI,  Baker  and 
Binford  81.  Mayhew  and  Frisby,  81],  which  provide  t !:re -him  nsional  positions  of 
sparse  sets  of  points  in  the  image.  This  sparse  data  can  be  used  to  reconstruct  a 
dense  surface  representation,  from  which  surface  normals  can  be  estimated  [Crimson 
82,  83;  Tcrzopoulos  83].  Other  methods  for  obtaining  three  dimensional  positions 
are  laser  range  finding  [e.g.  Nitzan,  Brain,  and  Duda  77,  Lewis  and  Johnston  77] 
and  structured  light  systems  [e.g.  Shirai  and  Suwa  71.  PoppleRone,  et  al.  75]. 
Many  other  visual  processes  can  be  used  to  obtain  surface  normal  information 
directly,  e.g.,  photometric  stereo  [e.g.  Woodham  78,  SO,  81.  lkeuchi  and  Horn,  79] 
and  texture  gradients  [Bajcsy  73,  Bajscy  and  Lieberinann  7C,  Render  80,  Stevens 
80].  In  fact,  there  is  no  constraint  that  the  sensory  data  for  one  problem  must  come 
from  one  sensory  modality.  Data  from  visual  sensors  and  tactile  sensors  may  be 
combined  in  one  run  of  the  algorithm. 

The  interpretation  stage  of  visual  recognition  has  received  less  attention, 
especially  when  dealing  with  three-dimensional  objects  with  six  degrees  of  freedom. 
Much  of  the  previous  work  in  the  area  of  interpretation  of  three-dimensional  data 
has  focused  on  tin  recognition  of  simpe  generic  objects  such  as  planar  patches, 
regular  polyhedra,  generalized  cylinders,  and  spheres  [e.g.,  Shirai  and  Suwa  71, 
Popplcstone,  et  al.  75,  Nitzan,  Brain,  and  Duda  77,  Oshima  and  Shirai  78, 
Faugcras,  et  al.  83,  Agin  and  Binford  73,  Nevatia  and  Binford  77].  Some  authors 
have  examined  the  problem  we  deal  with  here  or  recognizing  specific  objects  from 
three-dimensional  data  [e.g.,  Shncicr  79,  Sugihara  79,  Oshima  and  Shirai  83,  Holies, 
Horaud,  and  Hannah  83,  Brou  83,  lkeuchi,  ct  al.  S3].  The  principal  difference 
between  previous  work  on  recognition  and  the  approach  described  here  is  our 
reliance  on  sparse  data  acquired  at  points.  'This  makes  our  approach  adaptable  to 
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contact  sensing  as  well  as  visual  sensing.  7Tie  sparseness  of  the  data  does  make  the 
problem  of  segwe  ntation,  determining  which  data  is  drawn  from  which  objects  in 
a  scene,  more  dillicult.  Further  research  on  this  topic  is  currently  underway. 

In  the  final  stages  of  preparing  this  paper,  we  became  aware  of  the  work  of 
Faugeras  and  Hebert  [83]  which  adopts  an  approach  that  is  similar  in  many  respects 
to  the  one  described  here.  Their  work,  however,  focuses  on  deriving  an  accurate 
model  test.  Their  method  does  not  emphasize  the  problem  of  enumerating  all  the 
legal  interpretations  of  the  data.  Instead,  a  measure  of  the  accuracy  of  the  model 
test  (and  a  simple  angle  pruning  heuristic)  is  used  to  drive  a  best-first  search  for 
a  good  interpretation.  This  method  does  not  ensure  that  the  interpretation  found 
is  the  only  one  consistent  with  the  data,  however.  Their  method  and  ours  are 
complementary  in  this  respect.  Their  approach  also  does  not  assume  sparse  data, 
but  it  is  in  fact  applicable  to  t hat  problem. 

1.3.2.  Previous  Work  in  Tactile  Sensing 

Contact  sensors  measure  the  locus  of  contact  and  the  forces  generated  when 
in  contact  with  an  object.  Wc  make  the  distinction  between  tactile  seniors,  which 
measure  forces  over  small  areas,  such  as  a  fingertip,  and  force  sensors ,  which 
measure  the  resultant  forces  and  torques  on  some  larger  structure,  such  as  a 
complete  gripper.  A  micro-switch,  for  example,  can  serve  as  a  simple  tactile  sensor 
capable  of  detecting  when  the  force  over  a  small  area,  e.g.  an  elevator  button, 
exceeds  some  threshold.  The  most  important  type  of  tactile  sensors  are  the  matrix 
tactile  sersors,  composed  of  an  array  of  sensitive  points.  The  simplest  example 
of  a  matrix  tactile  sensor  is  an  array  of  micro-switches.  Much  more  sophisticated 
tactile  sensors,  with  much  higher  spatial  and  force  resolution,  have  been  designed; 
see  [Harmon  82]  for  a  review  and  [Hillis  82,  Overton  and  Williams  81,  Purbrick  81, 
Raibert  and  Tanner  82,  Schneiter  82]  for  some  recent  designs. 

For  descriptions  of  previous  work  in  tactile  sensing,  we  refer  the  reader  to 
two  very  thorough  surveys  by  Harmon  [80,  82].  A  more  detailed  discussion  of 
previous  work  on  tactile  recognition  can  be  found  in  [Gaston  and  Lozano-Perez  83]. 
In  this  section,  we  briefly  survey  the  two  major  alternative  approaches  to  tactile 
recognition:  statistical  pattern  recognition,  and  description  building  and  matching. 

Much  of  the  existing  work  on  tactile  recognition  has  been  based  on  statistical 
pattern  recognition  or  classification.  Some  researchers  have  used  pressure  patterns 
on  matrix  sensors  primarily  [Briot  79,  Okada  and  Tsuchiva  77].  Others  have  used 
the  joint  angles  of  fingers  grasping  the  object  as  their  data  [Briot,  Renaud,  and 
Stojilkovic  78,  Marik  81,  Okada  and  Tsuchiya  77,  Stojilkovic  and  Saletic  75].  A 
related  approach  uses  the  pattern  of  activation  of  on-olT  contacts  placed  on  the 
finger  links  [Kinoshita,  Aida,  and  Mori  75], 

The  range  of  possible  contact  patterns  between  multiple  sensors  and  complex 
objects  is  highly  variable  and  sccrns  to  require  detailed  geometric  analysis.  Tactile 
recognition  methods  based  on  statistical  pattern  recognition  arc  limited  to  dealing 
with  simple  objects  because  they  do  not  exploit  the  rich  geometric  data  available 
from  object  models. 
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Several  proposed  recognition  methods  buiid  a  pur*  .ie>i  r.plion  of  the  object 
from  t  he  sense  data  and  match  this  descript  ion  to  t he  mum  ».  Ora  approach  emulates 
the  feature-based  descriptions  in  vision  systems,  for  example,  men'.ficat  ion  of  holes, 
edges,  vertices,  pits,  and  burrs  [Ihnford  72,  I  bias  82,  Snydt ;  ami  St.  C  ’hi  r  78). 
Another  approach  is  to  build  surface  models,  either  from  pressure  distributions 
on  matrix  sensors  [Overton  and  Williams  SI  ,  or  from  tm  displacement s  (if  an 
array  of  needle-like  sensors  [Page,  Pugh,  and  Heginbot  ham  7b.  Takeda  7*lb  A 
related  approach  builds  a  representation  of  an  object's  cro^s  Minion  jOzaki  et  al 
82,  Kinoshita,  Aida,  Mori  75]. 

Description  based  methods  are  more  general  than  stnLsccn1  imthotis  but  must 
solve  two  formidable  problems:  building  accurate  obvci  ocvrrijC  > o i ; s  from  tactile 
data,  and  matching  the  descriptions  to  the  models.  Om-  majo:  dLbculiy  i>  that 
existing  sensors  do  not  have  the  spatial  or  force  resoiu: ion  i.erded  to  build  m  arly 
coniplete  object  descriptions.  Furthermore,  tlicre  are  iVw  me thuds  for  matching 
the  partial  descriptions  obtainable  from  tactile  sensors  to  object  models.  In  our 
opinion,  part  of  the  problem  in  tactile  data  interpretation  has  been  the  tendency 
to  adapt  the  techniques  developed  for  two-dimensional  vision,  where  dense  data  is 
readily  obtainable,  to  tactile  data,  which  is  naturally  sparse. 

One  lesson  from  the  simulations  desnibed  later  is  that  some  estimate  of  surface 
normal  is  an  extremely  powerful  constraint  o:i  meognit  ion  ami  localization.  The 
estimate  need  not  be  very  tight  for  performance  to  m:prov<  drastically.  1  here  has 
been  little  previous  emphasis  on  measuring  surface  wbh  tactile  sensors. 

Accuracy  in  measuring  normals  requires  some  attention  to  <  garnering  tradeoffs 
in  sensor  design,  especially  the  sensor  stillness.  Ir.  a  s i  ■  iV  soi.sor  (one  that  deforms 
very  little  under  contact),  the  normal  to  the  sensor  surface  at  the  point  of  contact 
directly  gives  an  estimate  of  the  object's  surface  normal.  So.  a  <ti ff  sensor  with  high 
spatial  resolution  can  be  used  to  measure  normals.  In  a  smt  sensor,  tiie  pattern  of 
forces  can  be  analyzed  to  determine  the  shape  of  the  object  surface.  So,  a  soft  sensor 
with  good  force  measurement  accuracy  can  also  be  used.  Today,  it  is  probably 
easier  to  build  stiff  sensors  with  poor  force  resolution  than  soft  sensors  with  good 
force  resolution  [Snyder  and  St.  Clair  78].  This  argues  that  a  stiff  VLSI  sensor  je.g. 
Raibcrt  and  Tanner  82]  may  be  acceptable.  Another  factor  is  that  the  method  used 
here,  since  it  is  based  on  local  information,  docs  not  require  large  sensor  areas;  it 
ran  function  better  with  many  small  sensors. 

The  approach  used  in  this  paper  is  an  instance  of  a  description  based 
recognition  method.  The  basic  departure  from  '  i<n..<  me!  bods  *:ie  reliance  on 
sparse  three  dimensional  positions  and  Miih’acr  .  *  ..  L  Me  aim  :  points1.  This 
contrasts  with  the  dense  cr  ’  lata  w  in  gJM  g  ;Va-ur-  -  al  or  surface  based 
iVscr:ption  methods.  1  hr  n  :a  oased  data  we  use  is  more  readily  obtainable 
from  simple  tactile  mnsors  arid  the  process  of  matching,  it.  to  models  is  relatively 
straightforward.  I  herefore,  the  method  described  here  could  be  a  powerful  addition 
to  approaches  based  on  more  complete  descriptions. 

'Very  different  approaches  to  tactile  recognition  based  on  t hi.-  tvpe  of  data  an  outlined  in  [Dixon, 
Salazar,  and  Slagle  19,  I vancc vie  74 J. 
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Figure  2.  Interpretation  Tree 


2.  Generating  Feasible  Interpretations 


After  sensing  an  object,  we  have  the  positions  of  up  to  5  points,  Pu  known  to 
be  on  the  surface  of  one  of  the  m  known  objects,  Oy,  having  n;  faces.  The  range  of 
possible  pairings  of  sensed  points  and  model  faces  for  one  object  can  be  cast  in  the 
form  of  an  interpretation  tree  (IT)  [Gaston  and  Lozano-Perez  83].  The  root  node 
of  the  IT;,  for  object  0;,  has  rij  descendants,  each  representing  an  interpretation 
in  which  Pi  is  on  a  different  face  of  Oy  There  are  a  total  of  s  levels  in  the  tree, 
level  1  indicating  the  possible  pairings  of  Pt  with  the  faces  of  object  0;  (see  Figure 
2).  Note  that  there  may  be  multiple  points  on  a  single  face,  so  that  the  number  of 
branches  remains  constant  at  all  levels. 

A  /c-interpretation  is  any  path  from  the  root  node  to  a  node  at  level  k  in  the 
IT:  it  is  a  list  of  k  pairings  of  points  and  faces.  The  set  of  IT’s  contains  a  very  large 
number  of  possible  s  -interpretations 

m 

£(»>)*• 

In  an  object  with  symmetries,  of  course,  the  IT  is  highly  redundant  [Gaston  and 
Lozano-Perez  83].  The  m  IT’s,  one  for  each  known  object,  represent  the  search 
space  for  the  recognition  problem  discussed  here. 

2.1.  Pruning  the  IT  by  Local  Constraints 

Only  a  very  few  interpretations  in  an  IT  are  consistent  with  the  input  data. 
We  can  exploit  the  following  local  constraints  to  prune  inconsistent  interpretations: 
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1.  Distance  Constraint  The  distance  between  each  .»  -ir  of  l\  s  must  he  a 
possible  distance  between  the  faces  paired  wnh  them  in  an  interpretation. 

2.  Angle  Constraint  The  range  of  po^ihh-  anvh*'  between  measured 

normals  at  eacii  pair  of  IV s  must  include  tin-  angle  between  surface 

normals  of  the  faces  paired  with  the;:-  u;  an  mti  rpi station. 

3.  Direction  Constraint  The  range  of  values  for  the  component  of  a  vector 

between  sensed  points  (/*  P3)  in  the  dincimn  c,f  the  sensed  normal  at 

I\  and  at  l\  must  intersect  the  ranee  of  components  of  possible  vectors 
between  points  on  the  faces  assigned  to  l\  ana  I1.  by  the  interpretation. 

These  constraints  typically  serve  to  prmu  most  of  the  non  sy mmetric  s 
interpretations  of  the  data.  Other  constraints  a:e  povbb'n  ,  for  example,  the  area  of 
the  triangle  defined  by  three  sensed  points  must  bi  amtainul  within  the  tango  of 
areas  defined  by  the  faces  paired  with  them,  ami  Die  panum  of  .mused  po  with 

faces  must  not  be  such  as  to  require  that  the  i>*t t : i  of  tne  v-n-or  (:v  urn)  : -a.  *ough 

some  portion  of  t fie  object  before  sensing  ! mv  fu<  e  on  and  Lozano-  cz  83]. 
We  will  focus  on  the  three  constraint^  above,  piuaar;  y  :x  masc  they  are  ,rie  to 

implement  while  being  quite  efTectivc.  Moreover.  May  cwnt  jir  all  the  c  mints 

between  pairs  of  points. 

Note  that  the  distance,  angle,  and  direction  cornu  ram;  s  can  be  used  io  prune 
^-interpretations,  for  k  >  2,  thereby  collapsing  whole  subtrees  of  the  IT.  This  is  a 
crucial  point,  worth  dwelling  on  for  a  moment. 

Recall  that  the  overall  problem  we  are  considering  to  (.Me;  mine  the  position 
and  orientation  of  an  object,  using  sparse  sensory  data,  in  principle,  one  could 
consider  all  possible  interpretations  of  the  data,  and  for  each  one,  determine 
whether  there  is  a  transformation  from  model  coordinates  to  suisor  coordinates 
that  would  account  for  the  sensory  data.  Unfuriunnw-b  .  tins  is  computationally 
extremely  expensive.  In  order  to  compute  such  a  model  test,  we  need  three  points, 
whose  corresponding  face  normals  are  iirmiriy  independent,  as  well  as  the  measured 
normals  at  those  points.  Clearly,  we  would  in  general  need  A"  sensory  points  to 
ensure  this,  where  k  >  3.  Thus,  if  n  is  the  number  of  faces  in  the  object,  we  would 
need  to  consider  on  the  order  of  nk  model  tests,  each  of  which  requires  considerable 
computational  effort. 

On  the  other  hand,  using  the  simple  geometric  constraints  outlined  above 
requires  only  a  straightforward  table  lookup,  and.  as  wc  shall  see,  can  drastically 
reduce  the  number  of  interpretations  to  which  a  mode!  test  must  be  applied.  Since 
the  constraints  can  be  applied  near  the  root  of  the  tree,  it  is  possible  to  prune 
whole  subtrees  from  the  IT,  at  virtually  no  computational  expense. 

Wc  consider  each  of  the  constraints  in  more  detail  below. 

2.1.1.  Distance  Pruning 

If  an  interpretation  calls  Tor  pairing  two  of  the  sensed  points  with  two  object 
faces,  the  distance  between  the  sensed  points  must  be  within  the  range  of  distances 
between  the  faces  (see  also  [Holies  and  Cain  82]).  Note  that  the  distances  between 
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all  pairs  of  sensed  points  must  he  consist « ■  n t .  Mo  tt  arc  three  distance 

three  sensed  points,  and  in  general  pj  detunes  between  /;  «  med  pi. mis.  M<  (  ,vum- 
of  this,  the  distance  constraint  typically  become.*  more  c!*:‘e<\.<*  as  mme  -msed 
points  are  considered. 

Given  two  faces  on  a  three-dimensional  obvit,  we  c an  Mu  rang''  of 

distances  between  points  on  the  faces.  Thi  minimum  disiamv  mav  be  det t  :  mined 
as  tiie  minimum  of  the  shortest  di>:.n.ee  bet  we*  n  all  pa::-  of  emu  cud  the 
perpendicular  abstain  cs  between  vertices  of  one  face  and  ‘Ur  piano  of  Mu  mimr 
face  (when  the  vertex  projects  inside  the  face  polygon).  T!a  maximum  requires 
examining  distances  between  pams  of  veince-u  Note  that  va  can  also  eomoute  the 
range  of  distances  between  points  on  o?k  face  \  zero  up  ‘o  Mu*  limun-r  the 
face).  Sophist  mated  algorithms  may  oe  used  to  redact-  the  •-omph  x.t  v  o'  thmt. 
computat  ions,  but  since  they  are  to  h<  performed  oil- line .  <-mv  :o:  each  mode-, 
their  eiliciency  is  not  cnlicai  to  ti:e  a[)proach. 

The  distance  constraint  can  be  implemented  in  the  following  manner  For 
object  Oj ,  with  fj  faces,  we  construct,  an  f,  by  f)  table,  whose  entries  determine  the 
range  or  possible  distances  between  pairs  of  faces.  In  particular,  for  a  pair  of  faces 
(t./c),  i  k,  the  maximum  distance  between  the  faces  is  stored  in  table  location 
(liable, f in ax(z,  fc),  min(i.  k)\  and  the  ntinirnum  distance  between  the  faces  is  stored  in 
table  location  citable,  rninp,  A'),  max(i.  A:)].  If  i  =  k ,  we  simply  store  the  maximum 
distance  in  the  diagonal  entry  citable^}?,  r.  since  the  minimum  distance  defaults 
to  0.  This  representation  makes  checking  a  distance  constraint  straightforward, 
since  the  set  of  all  pairs  of  faces  (i,k)  on  object  0;  consistent  with  some  measured 
distance  d  is  given  by 


(i,k)  |  dtablc;[inin(z,  fc),  max(z,  A*)]  <  d  <  citable,  jinaxp,  A  ),  inin(z.  k)\ 

plus  the  pair  (2,1)  if  d  <  dtable^ [i, 1]. 

Given  any  k  —  1  interpretation,  represented  by  the  set  of  faces  pi,  •  * 
and  a  new  kth  sensed  point,  the  generation  of  the  next  level  of  the  IT  below  this 
interpretation  can  be  easily  computed  by  checking  the  appropriate  portions  of  the 
distance  tables.  In  particular,  if  the  measured  distance  between  one  of  the  previous 
sensed  points,  i{,  and  the  new  one  is  given  by  dln  the  set  of  possible  faces  that  can 
be  assigned  to  sensed  point  is  given  by 


k~U 

pH  i  |  dtable;  [min(z,  it),  maxp,  2/)] 

*=1  ^ 


<  dt 


<.  dtable;  [max(i,  i(),  min 


unioned  with  the  set 


fl  {*<  I  0  <  <*.,  <  dtablcyft/,  if]). 

l= 1  1  J 

For  very  complex  objects,  much  more  time  efficient  ways  of  representing  and 
searching  for  faces  that  satisfy  a  distance  constraint  are  possible.  A  full  discussion 
of  these  methods  is  beyond  the  scope  of  this  paper,  however. 
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one  rot ational) .  Under  this  restriction  on  (i » r*  < ' ^  of  :re«co::u  1  he  range  of  surface 
nonnais  can  he  represented  as  a  range  o ;  an.rhs  .<  Vi  *o  •  n«  nan  i  fr.tme. 

At  a  sensed  point  i\  we  can  measure  the  iomh  su: face  no:;;. a.  as  !ymg  in  the 
ran se  of  angles 

o  G  —  c }  jj  c  j 

where  ^  is  the  actual  measurement,  and  t  dcniuw  nnr  :n:.o  of  pcssmie  .ingles  about 
tins  measurement.  We  are  given  a  sensor  roint  I\.  with  measured  normal  *,‘1,  which 
has  been  assigned  to  fact  g.  with  associated  mode!  coordinate  s^.rface  normal  given 
bv  I,.  Next,  we  record  a  second  point  /A*  with  measured  normal  ^2,  which  has 
been  assigned  to  face  An  with  associated  mode!  coordinate  surface  normal  given 
bv  L';.  For  these  assignments  to  be  consist  out.  it  must  be  the  case  that  the  angle 
between  the  model  faces  must  lx*  iuc!ud<  d  in  tin*  range  of  angles  between  the  ranges 
of  normal*  determined  from  the  measured  normals  and  the  error  bounds 

(^'2  —■  -m)  -  (< i  m*  <  >)  S  Vjt  —  t\  <  (a. 2  —  -m)  m-  (t  \  +  ^2)- 

It  ns  clear  that  an  implementation  similar  to  that  used  for  distance  pruning  will  also 
su'lice  here.  For  object  O, ,  with  r  1  edges,  we  can  set  up  an  f .  by  c7,  lower  diagonal 
table  atahlc;  s;wh  that  atable ,  max' : ,  A* h  m in; . .  A )  -  i.\  t/;.  I his  representation 

makes  checking  a  surface  normal  constraint  raigh.tforward.  since  the  set.  of  all 
pairs  of  faces  (z,  k)  c  1  object  Oj  consistent  with  some  measured  ranges  of  surface 
normals  is  given  by 

{M)l("2  —  uj  1 )  —  ( r  1  -f-tj)  <  alabIc;!max(;U).  inin(r,  k)}  <  (utj -  - t*'i)  “Mfi  +  O 
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Figure  3.  Angle  Ranges 
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Given  any  k —  1-interprotation,  and  a  new  kth  sensed  point,  the  general  ion  of  the 
next  level  of  the  IT  below  this  interpretation  can  easily  hr  computed  by  checking  the 
appropriate  portions  of  the  angle  tables.  Note  that  the  k‘  ‘  edge  must  be  consistent 
with  the  angles  between  all  previous  faces. 

In  the  two-dimensional  (three  degree  of  freedom)  case,  the  range  of  possible 
surface  normals  at  a  sensed  point  was  represented  by  the  pair  (x'i.g)  where 
denoted  the  sensed  normal,  and  ( \  denoted  the  range  of  error  about  that  sensed 
point.  In  three  dimensions,  the  obvious  generalization  is  to  use  angle  cones,  so  that 
if  U]  denotes  the  unit  sensed  surface  normal,  the  range  of  possible  values  for  the 
actual  surface  normal  will  be  denoted  by  the  right  circular  cone 

{«i  !  m  ■  uj  >  fj}. 

Wc  could  proceed  identically  to  the  two-dimensional  case  by  noting  that  the  cone  of 
sensed  normals  constrains  the  set  of  possible  three-dimensional  rotations  between 
the  hand  and  model  coordinate  systems.  Then,  given  a  second  sensed  point.  P2  with 
some  sensed  normal,  the  sm  of  feasible  faces  would  oe  restricted  by  the  range  of 
possible  rotations.  This  method  is  quite  difficult  to  implement,  however.  There  is  a 
much  simpler  alternative  method. 

Suppose  that  at  the  second  sensed  point,  the  set  of  possible  surface  normals  in 
hand  coordinates  is  given  by 

{n2  |  n2  •  u2  >  <2}- 

Then,  in  order  for  faces  1  and  ky  with  associated  surface  normals  vt  and  to  be 
consistent  it  must  be  the  case  that 

v*  f  { n !  •  n2  |  ni  •  uj  >  t\,  n2  ■  u2  >  f2}- 

We  can  rephrase  this  in  the  following  manner.  Let  cosai  <  i<  cosao  =  r2, 
f>i2  —  at\  ~f-  a2  and  cos*>i2  ~  u \  •  u2.  Then,  wc  churn  that  the  set 
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{nj  ■  n2  i  ill  ■  Ui  >  Ci,  n2  U2  >  t>} 

is  contained  in  the  set 

{n i  no  cos  [min(7rt  ^12  o  12);  ^  ni  •  n2  ^  cos  [max(0.  712  —  a  12)1}-  (l) 

A  proof  of  this  is  found  in  Appendix  I.  Future  3  illustrates  mis  result  m  two 
dimensions. 

An  implementation  of  angle  pruning  similar  to  t hat  used  for  distance  pruning 
is  now  also  possible.  For  object  0)%  with  f,  faces,  we  can  sot  up  an  fj  by  fj ,  lower 
diagonal  table  atablc,  such  that  atablc  1  max(i.  k).  rnin(?.  k)]  =  v;  •  \\-,  where  recall 
that  v \  denotes  the  unit  normal  to  face  1  in  the  model. 

2.1.3.  Direction  Pruning 

Consider  a  pair  of  sensed  points  P\  and  P2  and  let  ul:  be  the  unit  direction 
vector  between  them.  Suppose  that  we  know  the  measured  surface  normal  at  point 
P\  to  within  some  cone  of  error,  for  example,  the  measured  value  is  wlt  and  the 
range  of  possible  values  for  the  surface  normal  is 

{'1  !  vt  •  wj  >  «,}• 

Then  the  set  of  possible  “angles'  between  the  direction  vector  and  the  surface 
normal  of  the  face  is  given  by 

{V,  •  U12  ;  V,  W,  >  f  l}. 

In  an  interpret  a*  ion,  suppose  that  point  P\  has  been  assigned  to  face  2,  with 
norma!  nz  in  the  model,  and  we  now  consider  possible  faces  k  to  assign  to  point  Pn , 
Let  tiie  range  of  possible  unit  vectors  (directions)  from  fare  1  to  face  k  be  denoted 
by  the  cone 

{*.<■  I  ■  t,k  <*U} 
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for  some  pair  t,„  and  dlK-  Figure  1  illustrates  tins  rone  in  a  two  dimensional 

example.  Appendix  11  shows  how  this  rone  may  be  computed  from  models  of’  the 

object  faces.  In  the  model,  the  set  of  possible  angles  between  leg.d  ( !  j :  *  *  r  t  ions  and 
the  surface  normal  is 

{ip  siV  Sl’<  t tk  N  (^j 

Thus,  assume  that  point  l\  is  on  fact  ».  with  normal  rq,  that  we  have  measured 
Wi,  that  we  know  and  that  we  have  also  measured  /V  A  face  k.  whose  direction 
range  from  fare  i  is  given  by  the  pair  ns  a  feasible  face  for  point  !\  F  the 

set.  in  equation  (2)  intersects  the  cone 

{'  i  n i2  1  v i  wi  >  <i}.  (3) 

If  cos  ----  <\  a-  ,  and  coso:jt  iq  t,^.,  then  we  know  from  the  dm.;.:  or.  r  Anpcmbx 
1  that  the  set  of  equation  (2)  is  contained  in  the  set 

{n*  Sik  |  eos(-) lfc  f  S  n:  •  si\  ros(  :ik-  ~ 

Similarly,  if  cosoj  =  t\  and  rosu;^  vi  ui:-  then  the  set  of  equation  (3)  is 
contained  in  the  sot 

{vj  •  u j  ?  |  cos(tti  +  <  vj  •  u 2 2  £  eos(nj  —  ^1:)}- 

Therefore,  for  the  pairings  of  ]\  with  face  1  and  t\  with  face  k  to  t,e  consistent  with 
the  direction  constraint,  it  must  be  the  ease  that  the  intersection  of  the  numerical 
ranges  of  dot  products  is  not  null,  i.e., 

' cos(ni  —  ^v>).  cos(a t  —  _•;_>)•  0  [cos(7t*  —  Oik),  ros( 7,4  -f  otk)}  0 

The  direction  constraint  can  also  be  implemented  in  a  form  similar  to  that  used 
for  distance  and  angle  pruning.  For  object  Or  with  f1  faces,  we  can  set  up  an  fj 
by  f  ,  table  ctahlc,  such  that  rtahle  , \i,  k]  — [cos^^  —  &:k)'C0*{~Uk  “t"  Again, 

the  set  of  ai!  pairs  of  faces  (i,  k)  on  object  01  consistent  with  some  measured  ranges 
of  surface  normals  is  given  by 

(<\  k)  ,  ;cos(ai  —  ^12).  cos(aj  4-  ^l2  )j  n  ctablej  [1,  k)  =£  0 

Note  that  the  direction  constraint  is  not  symmetric,  as  arc  the  distance  and 
angle  constraints,  so  before  pairing  I\  to  face  k,  we  must  repeat  the  test  above 
interchanging  the  roles  of  1  and  k.  Similarly,  the  test  must  be  applied  to  each 
pairing  of  sensed  points  and  faces  in  an  interpretation. 

The  constraint  described  above  places  constraints  on  the  angle  between  a  surface 
normal  and  unit  vectors  from  one  face  to  another.  In  addition  to  constraining  the 
angles  of  unit  vectors,  we  may  constrain  the  magnitude  of  the  component  along 
the  surface  normal  of  the  vector  between  the  sensed  points.  The  statement  and 
implementation  of  the  constraint  is  essentially  unchanged,  except  that  U12  and  ttfc 
are  no  longer  unit  vectors  but  the  actual  vector  between  the  sensed  points.  The 
effectiveness  of  the  constraint  is  in  general  improved,  however,  since  it  nowr  captures 
some  distance  and  some  angular  constraint.  The  difference  between  this  extended 
direction  constraint  and  the  simple  direction  constraint  is  illustrated  in  Figure  5. 
Two  parallel  faces  (faces  1  and  2  in  the  figure)  displaced  relative  to  each  other 
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Figure  5.  Extended  Direction  Const.  it 
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give  rise  to  a  cone  of  directions,  but  a  single  value  for  the  norma!  component  of 
vectors  connecting  the  faces.  Note  that  an  interpret atat :on  that  assigns  I\  to  face  1 
and  P 2  to  face  3  is  consistent  with  all  the  previously  nu-nt  ioned  constraints  except 
for  the  extended  direction  constraint.  The  figure  also  :!iust rates  that  the  extended 
direction  constraint  does  not  subsume  the  distance  ton: : rami .  muw  uirec.tion  only 
constrains  the  normal  component  of  distance. 

There  is  an  alternate  form  of  the  direction  cons; ram*. ,  useful  when  no  bound 
on  the  surface  norma!  is  available.  It  can  briefly  be  described  as  follows,  (liven  two 
faces  h  and  i  on  an  object,  we  ran  compute  the  range  of  dirocHons  between  points 
on  the  faces,  forming  a  cone  of  possible  directions.  Niuilaiiy  for  faces  i  and  j}  we 
can  compute  the  cone  of  possible  directions.  The  combination  of  these  two  cones 
defines  a  range  of  possible  angles  for  the  triplet  of  fares  h.i,  j. 

If  an  interpretation  calls  for  pairing  three  of  the  sensed  points  with  three  object 
faces,  the  angle  formed  by  this  triplet  of  sensed  points  must  he  within  the  range 
of  possible  angles  between  the  triplets  of  fares.  Note  that  the  angles  formed  by  all 
triplets  of  sensed  points  must  be  consistent,  i.t .  for  three  sensed  points,  there  are 
three  angles,  for  k  sensed  points,  there  are  3Uj  angles.  Hence,  this  constraint  also 
becomes  more  effective  as  more  sensed  points  are  considered. 

This  form  of  the  direction  constraint  ran  be  nsec!  when  only  vertices  and  edges 
are  touched,  as  it  does  not  require  sensing  surface  normals.  Note  that  this  form  of 
the  constraint  can  also  be  extended  to  use  the  magnitmh  of  the  vectors  between 
sensed  points  as  well  as  their  direction.  This  form  </  tin  d/vc:  'm  constraint  allows 
pruning  of  the  IT  foi  k  >  3.  The  previous  formulation  of  the  constraint  allows 
pruning  of  the  IT  for  k  >  2.  As  well,  this  form  of  the  comm r. out  would  require  an 
rr*  table,  as  opposed  to  an  n2  one  for  the  previous  formula*. ion.  Given  the  size  of  n 
to  be  expected  for  typical  objects,  this  is  a  critical  difference. 
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3.  Model  Testing 

Once  the  interpretation  tree  has  been  pruned  by  the  local  constraints,  there 
will  be  some  set  of  possible  interpretations  of  the  sensed  data,  each  om-  consisting 
of  a  set  of  triples  (|>j,  n:, /,),  where  p,  is  the  vector  representing  the  screed  position, 
nI  is  the  vector  representing  the  sensed  normal,  and  ft  is  the  face  assigned  to 
this  sensed  data  for  that.  particular  interpretation.  In  the  model  test  stage  of  the 
processing,  we  want  to 

1.  determine  the  actual  transformation  from  model  coordinates  to  sensor 
coordinates,  corresponding  to  the  interpretation, 

2.  check  that  under  this  transformation,  not  only  are  the  sensed  [joints 
transformed  to  lie  on  the  appropriate  planes,  but  moreover,  lira  the 
sensed  points  actually  lie  within  the  bounds  of  the  assigned  faces. 

We  will  assume  that  a  vector  in  the  model  coordinate  system  is  transformed 
mto  a  vector  in  the  sensor  coordinate  system  by  the  following  transformation: 

V5  =  RVm  +  V0 

where  R  is  a  rotation  matrix,  and  vo  is  some  translation  vector.  We  need  to  solve 
for  R  and  vq.  We  note  that  a  solution  could  be  obtained  using  a  least-squares 
method,  such  as  is  used  by  [Faugeras  and  Hebert  83].  This  type  of  solution  can  be 
computationally  expensive,  however,  and  in  the  following  sections,  we  develop  an 
alternative  method. 

3.1.  Rotation  Component 

We  consider  first  the  rotation  component  of  the  transformation.  Consider  the 
first  triple  of  a  particular  interpretation,  (p,,  rq,  f%).  The  sensed  normal  is  given  by  vt 
and  corresponding  to  face  ft  is  a  face  normal  mt.  For  R  to  be  a  legitimate  rotation, 
it  should  take  the  normal  mt  into  ut  (ignoring  issues  of  error  in  the  measurements 
for  now). 

Now,  any  rotation  can  be  represented  by  a  direction  about  which  the  rotation 
takes  place,  and  an  angle  of  rotation  about  that  direction.  What  is  the  set  of  possible 
directions  of  rotation  r  consistent  with  n,  and  mt?  Any  rotation  will  preserve  the 
angle  between  the  transformed  vector  and  the  direction  of  rotation.  Hence,  any 
legitimate  rotation  direction  must  be  equiangular  with  nt  and  nq.  Thus,  the  set  of 
potential  directions  is  given  by 


or  equivalently 


That  is,  TtJ  is  perpendicular  to  (m,  —  nt). 
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Now,  consider  a  second  triple  in  the  interpret  a:  ioi  ;r.n7./;)  arid  let  m;  be 
the  normal  to  face  f3.  Provided  m;  ^  imi  hikI  n,  m.  is  not  (anti-)parallel  to 
n;  —  \n]t  we  can  constrain  rl3  to  a  second  set 

jiy,  |  rtJ  •  (in,  —  n;)  =  o|. 

Since  the  rotation  is  the  same,  r,;  must  he  in  both  sets,  i.e.,  it  must  be  perpendicular 
to  both  vectors.  lienee,  rtJ  is  given  by  the  unit  vector  in  the  direction 

(m,  —  n,)  X  [ntj  —  n,) 
to  within  an  ambiguity  of  180°. 

'['his  derivation  can  be  recast  in  geometric  terms  in  tin  following  manner.  Any 
unit  rotation  vector  r  taking  in,  into  nl  must  lie  on  the  perpendicular  bisector  of  the 
line  connecting  n,  to  m2.  Similarly,  it  must  also  lie  on  the  perpendicular  bisector 
of  the  line  connecting  n^  to  m;.  Since  the  rotation  is  the  same,  it  must  lie  in  the 
intersection  of  the  two  perpendicular  bisector  planes,  as  above,  and  hence  is  given 
by  the  specified  unit  vector 

(m,  —  n,}  X  (in,  —  n,). 

If  there  were  no  error  in  the  sensed  normals,  we  would  be  done.  With  error  included 
in  the  measurements,  however,  the  computed  rotation  direction  r  could  be  slightly 
wrong.  One  way  to  reduce  the  effect  of  this  error  is  to  compute  all  possible  rt)  as 
z  and  j  vary  over  the  faces  of  the  interpretation,  and  then  cluster  these  computed 
directions  to  determine  a  value  for  the  direction  of  rotation  r. 


Once  we  have  computed  a  direction  of  rotation  r.  we  need  to  determine  the 
angle  0  of  rotation  about  it.  It  is  straightforward  to  show  that  (see,  for  example, 
(Korn  and  Korn,  68]  p.  473) 

rnt  =  cos  0n,  -f  (1  —  cos  0)(r  n2)r  -J-  sin  0( r  X  nt). 

Simple  algebraic  manipulation,  using  the  fact  that  r • in,  ~  r  nt,  yields 


cos  0=1 


sin  9  = 


1 


__  1  —  (n,  •  in,) 

1  —  (r  ■  n,j(r  ■  m,j 
(r  X  n,)  •  m, 

—  (r  •  n,j(r  •  rn,j 


Hence,  given  r,  we  can  solve  for  0.  Note  that  if  sinfl  is  zero,  there  is  a  singularity 
in  determining  0 ,  which  could  be  either  0  or  k.  In  this  case,  however,  r  lies  in  the 
plane  spanned  by  n,  and  m,  and  hence,  only  the  0  — -  r  solution  is  valid. 


As  before,  in  the  presence  of  error,  we  may  want  to  cluster  the  r  vectors,  and 
then  take  the  average  of  the  computed  values  of  0  over  this  cluster. 


Finally,  given  values  for  both  r  and  0 ,  we  can  determine  the  mUtum  matrix 
R.  Let  rx,ry,rz  denote  the  components  r.  Then 


‘1  0  O' 

'  r\  rxTy  rxr / 

0  -  r2  Ty 

0  1  0 

-f  (1  —  cos  6) 

ryrt  r2  ryr, 

i  SI  1)0 

|  0  Tx 

.0  0  1. 

r2rx  rzTy  r\  _ 

[■  n  r*  0  . 
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Note  that  in  computing  the  rotation  component  of  the  transformation,  wo  have 
ignored  the  ambiguity  inherent  in  the  computation.  Thai  is,  there  are  two  solutions 
to  the  problem,  (r,  0)  and  ( —  r,  —  0).  We  assume  that  a  simple  convent  ion  concerning 
the  sign  of  the  rotation  is  used  to  choose  one  of  the  two  solutions. 

3.2.  Translation  Component 

Next,  we  need  to  solve  for  the  translation  component  of  the  transformation. 
We  know  that  \s  R\m  -j  vq,  where  \rn  is  a  vector  in  model  coordinates,  vs  is  the 
corresponding  vector  in  sensor  coordinates,  and  It  has  been  computed  as  above, 
(liven  a  triple  (plti from  the  interpretation,  let  rip  be  the  normal  of  face  /lf 
with  offset  dlt  that  is,  the  face  is  defined  by  the  set  of  vectors 

{v  |  V  •  Ill,  =  dt}. 

Then  the  point  in  model  coordinates  corresponding  to  p:  is 

Ft “‘(p,  —v0) 

and  the  following  equation  holds 

mt  •  —  v0))  =  dt 

or  equivalently 

{Ft m.)  •  (p,  —  v0)  =  dx. 

This  equation  essentially  constrains  the  component  of  the  translation  vector  in  the 
direction  of  R mt. 

Suppose  we  consider  three  triplets  from  the  interpretation,  (p,,  nn  /t),  (p;,  n;,  /y), 
and  (pfc<n/c>  fk)  such  that  the  triple  product  •  (m;  X  nijt)  is  non-zero,  (i.e.  the 
three  face  normals  are  independent).  Then,  we  can  construct  three  independent 
equations 

(ftm.)  •  v  o  =  (Rm,)  p,  —  dt 
(Ftm})  ■  v0  =  •  py  —  dj 

(Hmjt)  •  v0  =  ( Rmk )  •  p*  —  dk. 

Each  of  these  equations  constrains  a  different,  independent  component  of  the 
translation  vector  vq,  and  hence  the  three  equations  together  determine  the  actual 
vector.  Straightforward  algebraic  manipulation  then  yields  the  following  solution 
for  the  translation  component  vo: 

mt  ■  (m,  X  in*)Jv0  =((/?m,)  •  p,  —  <*,)((/?">,)  X  [Rmkfj 

+  '  Pj  —  dj)iif{mk)  X  ( Rm, )) 

+  pm*)  •  P*  —  dk)((Rmt)  X  (Ftm y)) 

As  in  the  case  of  rotation,  if  there  is  no  error  in  the  measurements,  then  we  arc  done. 
The  simplest  means  of  attempting  to  reduce  the  effects  of  error  on  the  computation 
is  to  average  vq  over  all  possible  trios  of  triplets  from  the  interpretation.  Note  that 
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for  numerical  stability,  one  may  want  to  reMrici  tin*  .  puta'am  to  triplets  sueh 
that  int  •  (in;  X  in*)  ls  greater  than  >orne  threshold. 

Finally,  we  have  computed  the  transform  (/i\v;j)  f ■  worn!  coordinates 
to  sensor  coordinates.  To  check  a  possible  interpretation.  \v<  omMder  all  triples 
lPono/i)  the  interpretation  and  compute 

R  '(p,  -  v0). 

We  then  check  that  this  point  lies  within  the  bcm:;<;>  of  hm*  /  (to  within  some 
error  range).  If  it  does  not,  then  the  interpretation  ;>  mvami.  .n  la.-y  be  pn  ,,d. 
If  all  such  triples  satisfy  this  check,  the  interpretation  ’*>  st . ; 1  valid 

We  have  assumed  above  that  three  mbepciuh  :.r  five  ’mm  ,,X  have  been 
measured.  When  only  one  normal  is  available,  the  :•  w  :  o’  ‘  ranslat.on 

can  be  determined.  When  only  two  independent  % i r -  an  ...,;A:hh\  the  rotation 
ran  be  determined  as  before,  but  only  a  din  mum  of  ir.msla1  •"*  .  ;.•»  b<  determined, 
not  the  actual  magnitude  of  the  translation.  A  :a;.m  e*  j  ‘ :  •, miniums  can 

be  dt  UTinmed,  howe\«r,  by  interesectiug  tin*  km.  bi :  «•:  warn  .  ‘h<-  poMtion  of  a 
sensed  point  and  the  translation  direction,  will;  »!. e  .. meo  ;o  t:u*  point  by 

the  interpretation.  Of  course,  further  sensing  along  thm  :nu  to  d>co\a  r  the  position 
of  the  edge  would  determine  the  actual  translation. 

After  the  model  tost  has  been  applied  to  ail  leaver  of  tiu  interpretation  tree, 
there  may  still  be  several  interpretations  remamim’  mm  ex.?  ■  .nation,  one  usuallv 
finds  that  these  interpretations  differ  only  m  Tm  •.  ;m  <e  cm*  or  two  faces, 
all  other  faces  being  identical.  This  inability  u  X:  ..  o- k  w  .‘.ecu  mu  h  nearly 


identical  interpretations  is  a  result  of  the  error  b  amis  on  tl.i  sensing.  Thus,  as 
a  final  stage,  we  duster  the  remaining  interpretations  m  terms  X  their  computed 
transformations,  that  is,  we  cluster  the  interpretation'*  in  ;•  s  >f  the  computed 
orientation  of  the  object  in  space.  Here.  w<  giimiAk.  tmd  v«  \  ;*  v.  m  u  clusters. 
Indeed,  m  general  there  is  only  one  computed  orient  atm;;  for  eb;ecr,  ft  he  correct 
one),  although  occasionally  two  or  more  clusters  survive,  umm  y  corresponding  to 
symmetric  interpretations  of  the  sensed  data. 


1.  Simulation  Data 

In  order  to  test  the  efficacy  of  the  algorithm  m  pruning  -ho  interpretation 
tree,  \vi*  ran  a  large  number  of  simulations.  Some  simuhthoim  for  objects  with  three 
degrees  of  freedom  (two  translational  and  one  rot  at  am. d)  have  iven  described  in 
(iasion  and  Lozano- Perez  83].  We  include  additional  Simula' .on  data  for  objects 
with  three  positional  freedoms,  including  the  direction  mnslrami.  W«  also  provide 
data  for  the  more  general  case  of  three  dimensional  ^joc -  mi;  v\  degrees  of 
freedom. 

Our  goals  are  first  to  demonstrate  that  effective  prumt.e  of  the  interpretation 
tree  is  possible,  at  low  computational  expense,  and  second  fo  exphm  the  <ruMbvily 
of  the  algorithm  to  error  in  measuring  the  surface  normal  amt  Xu  posiXon  of  the 
sensed  points. 
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4.1.  Three  Positional  Freedoms 

We  begin  by  considering  objects  with  two  degrees  of  translational  freedom 
and  one  degree  of  rotational  freedom,  using  sample  objects  first  considered  m 
[Gaston  and  Lozano-Perez  83],  illustrated  in  Figure  6.  1'he  addition  of  the  direction 
constraint  greatly  reduces  the  extent  of  the  set  of  possible  interpretations.  To 
demonstrate  this,  a  series  of  250  runs  of  the  algorithm  was  executed  for  each  of  the 
objects.  Each  run  determined  the  number  of  interpretations  consistent  with  a  set  of 
5  sensed  points.  The  points  were  determined  by  first  randomly  rotating  the  object 
about  its  centroid  and  then  intersecting  the  object  with  five  lines  from  its  centroid 
along  five  evenly  spaced  directions.  The  points  of  intersection  farthest  from  the 
centroid  along  each  line  were  used  as  the  sensed  point.  The  (simulated)  error  in 
measuring  the  sensed  position  was  bounded  by  0.1  (i.e.  a  randomly  oriented  offset 
vector  of  random  magnitude  bounded  by  0.1  was  added  to  the  point  on  the  object), 
and  the  (simulated)  error  in  measuring  the  angle  of  the  surface  normal  was  |  (i.e.  a 
random  vector  was  chosen  whose  dot  product  with  the  actual  normal  was  bounded 
by  cos“*  |).  To  place  these  error  ranges  in  perspective,  the  diameters  of  the  models 
in  Figure  6  were  9,  14  and  12  units  for  the  wrench,  gator  and  hand  respectively. 

The  following  table  describes  the  results  of  this  set  of  simulations,  by 
histogramrning  the  number  of  interpretations  found.  Thus,  for  i  <  10,  the  number 
in  the  itfl  column  is  the  number  of  trial  runs  which  resulted  in  i  possible 
interpretations.  Beyond  this  point,  the  histogram  is  compressed  into  units  of  tens. 
For  example,  the  column  labelled  20  lists  the  number  of  trial  runs  resulting  in  k 
interpretations,  where  10  <  k  <  20.  In  order  to  examine  the  effectiveness  of  adding 
the  direction  constraint  to  the  algorithm  described  in  [Gaston  and  Lozano-Perez 
83],  the  simulations  were  run  both  with  and  without  this  constraint.  For  each  object 
in  the  table,  the  first  histogram  corresponds  to  the  case  of  using  the  direction 
constraint,  and  the  second  histogram  to  the  case  of  not  using  it.  Note  that  the 
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number  of  edges  for  the  wrench  (\V),  gator  ((i)  and  .  mi  II)  is  1 2.  30  and  67 
respectively. 
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maximum  number  of  possible  interpretations  ohsi-rvib  fo:  any  < ■:  im  objects  was 
20  (m  the  case  of  using  the  direction  constraint),  wmm,  ;  <  .•:*•.  ;onn!iy  low  w  hen 
considering  that  the  total  number  of  possible  mteprm  m  a  •  -  for  the  gator  was 
30:>  or  312,300,000.  Second,  the  median  number  of  pu>*;:  •  ' * t prm  atious  was 

only  2  for  the  wrench,  and  \  for  the  gator  and  ham:,  w.  e;:  t.s.ag  *  he  direction 
constraint.  Without  Fui>  constraint,  the  mcumn  nuu.i.'c:  m‘  m’orpretations  rose  to 
•18,  12  and  9  for  the  wrench,  gator  and  hand,  respect iveiy .  Of  course,  the  results  of 
the  simulations  will  depend  to  a  certain  extent  on  tin  error  ranges,  a  point  that 
will  be  explored  in  some  detail  in  the  next  section.  \\V  not e  that  a  tenth,  of  an  inch 
sensitivity  m  distance  over  a  10  to  20  inch  ramp.  >  within  hue  range  of  cu  rent 
tactile  sensors.  The  positioning  accuracy  of  mmr,  n  u  .iato;  s  is  within 

0.01  inches,  and  the  Purbrick  tactile  sensor  has  a  matiix  cieiium  separation  of  0.06 
inches,  and  the  Milhs  sensor  has  an  element  separation  of  0.023  inches. 

1.2.  Six  Positional  Freedoms 

When  considering  the  full  three-dimension, d  proniem  . » f  objo.  :.s  with  six  degrees 
of  freedom,  we  have  run  extensive  simulations  on  me  modem  l  i  mi  rated  m  figure  7. 
The  diameters  of  these  objects  (that  is  the  maximum  ^pmatiou  of  two  points  on 
mu  object)  were  roughly  *1,  7,  8  and  8  inches  for  the  horn;:;,.;,  stapler,  simple  hand 
and  complex  hand  respectively.  In  running  simulations  of  \‘.c  .million  algorithm 
on  these  objects,  we  have  used  two  different  sensing  si ;  m  i  gam  reilecluig  in  part 
the  difference  between  range  and  tactile  sensing  capabilities. 

It  should  be  noted  that  in  all  the  following  simulations,  the  efficiency  of  the 
tret'  pruning  mechanism  was  improved  by  sorting  the  sensed  points.  In  particular, 
rather  than  using  the  sensory  data  in  arbitrary  order,  -  hr  potiim  were  sorted  on  the 
basis  of  pmrwme  separation,  with  the  more  distant  points  being  ordered  first.  This 
s»rtmg  on  distance  tends  to  place  the  most  rilVcuvv  constraints  ;C  the  beginning 
of  the  process,  a  pom  I.  that  will  be  illustrated  m  Section 

T3.  (irid  Sensing 

In  the  first  sensing  method,  the  sensory  data  were  generated  by  projecting  a 
regular  grid  of  points  along  three  orthogonal  directions,  and  noting  where  contact 
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whs  made  with  an  arbitianly  oriented  model  of  ihe  objee*  '  :  i:  rar v  mwm  at  ion 

was  obtained  by  ramiomly  choosing  values  for  the  ii:n-r  mm-"  angle*,  <  ornputing 
a  rotalion  trausforrnat  ion  bast'd  on  this  and  applying  m  nmmrni  ;o  the  model. 
Note  that  tins  tioes  not  produce  a  uniform  samphm. i  of  m<  sp.-iw  of  ;•</,  a*  ions,  but 
for  our  purposes  it  is  a  stiilinent  ly  rando:u  sampling.  No  * :  amln:  :o::  <dT-r*t  was 
added,  since  this  would  no?  affect  the  proofs.  I  ne  mree-b  :;.i  imonai  pom?  ions  of 
the  sensed  points  anti  the  associated  surface  normah  v><  re  mm:  curru  e.i  b\  noise 
Within  some  specified  bounds.  For  the  simulations  discus>ed  b«  ww.  t;,e  mimoer  of 
sensed  points  on  each  trial  varied  between  12  and  20. 

I  he  results  oi  the  iirsf  s»*t,  ol  Simula.,  ms  >  show  i.  v.  !.-,n.»*>  11.  lip  anti  IV. 
1'able  II  lists  staUs’ics  of  *he  number  of  u.terpr*  *  ai  au.^  ■;  ,i  *  :<  <.  fwhowm^  local 
pruning,  fora  variety  of  <e  using  accuracies.  Mach  M?:.i.ia:  ;o:.  >  t '  < :  of  ;(}()  t  rials, 

and  the  miminum  mu’*  maximum  number  of  intcrpnUa:  mm  ;i  embed  over  this 
set  of  trials,  as  well  as  tin*  aOt  h  and  OUt  i.  p»rce;n;ie  of  da  u; ;•;:»■*  t  ion  of  number 
of  interpretations,  Table  ill  lists  statistics  of  tin'  number  of  am  :  :>n  :  a* urns  m  the 
IT  that  survive  an  explicit  mode;  n*s:.  It  was  observed  ;0  mm  - : . ; ;  e  ‘a.  it  while  the 
number  of  interpre:  muons  was  not  reduced  to  i.  as  niich-  n*  •  am  *<  d  ::  <  o.rvirmg 
interpretations  generally  leaded  :.o  differ  only  in  (*■:;«•  w  two  Moicover,  the 

computed  transformation  parameters  were  neariy  a. mu a... h  aumra  Mum  the 
multiple  interpretations  surviving  a  model  test  acmnhv  cor:e> ponded  to  single 
interpretation,  to  within  the  error  ranges  of  the  aLor.thw.  ’inis.  Tab!-  IV  lists 
statistics  of  the  number  of  separate  transform;:?  mm*  mm  *  u  :  f ■ : ;  e  .m.  mai.  In 
particular,  transformations  whose  direction  of  nuamu  :::!;  sc  ,  .  re  1 . 5° 

were  judged  to  be  dilTercnt,  yielding  a  very  tight  (lusmrmg  ihe  ■  run  puled 
transformations.  This  clustering  ignores  differences  in  me  translation  component, 
a  point  that  is  addressed  later  in  Table  VI. 
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In  the  table  above,  the  normal  column  lists  the  radius  of  the  error  cone  about 
the  measured  surface  norma!;  the  dist  column  lists  the  error  range  of  the  distance 
sensing;  the  min  and  max  columns  list  the  minimum  and  maximum  number  of 
interpretations  observed;  the  50th  column  lists  the  median  point  of  the  set  of 
simulations;  the  90th  column  lists  the  90r*  percentile  of  the  set  of  simulations;  and 
the  faces  column  lists  the  number  of  faces  in  the  model. 
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In  the  tabic  above, 

the  normal  column  lists  the 

radius 

of  the 

error  cone 

about 

the  measured  surface  normal;  the  dist  column  lists  the  error  range  of  the  distance 
sensing;  the  mm  and  max  columns  list  the  minimum  and  maximum  number  of 
interpretations  observed;  the  50th  column  lists  the  median  point  of  the  set  of 
simulations;  the  90th  column  lists  the  90th  percentile  of  the  set  of  simulations;  and 
the  faces  column  lists  the  number  of  faces  in  the  model. 
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Figure  8.  Simple  Hand  Histogram* 


The  first  point  to  stres$  is  that  all  of  these  numbers  are  remarkably  low.  given 
that  the  total  number  of  possible  interpretations  of  15  sensed  points  on  an  object 
with  10  faces  is  roughly  1.07*1  X  1024.  Thus,  the  local  geometric  constraints  are 
very  effective  in  reducing  the  combinatorics  of  feasible  interpretations. 

As  might  be  expected,  the  number  of  interpretations  in  all  three  tables  tends  to 
ris“  with  increasing  error  in  the  measured  parameters.  The  distributions  also  tend 
to  be  strongly  clustered  near  the  low  end  of  the  scale,  with  a  very  shallow  tail  on  the 
high  end  of  the  distribution.  Thus,  while  the  maximum  number  of  interpretations 
can  be  high  (e.g.  3576  for  surface  normal  error  cone  of  -/8  and  distance  error  of 
CUn).  the  median  point  and  even  the  90th  percentile  of  the  distribution  are  generally 
much  smalle  r.  Sample  distributions  for  the  number  of  interpretations  surviving  tree 
prim im:  are  shown  m  Figure  8.  One  reason  that  the  maximum  number  of  feasible 
interpretations  can  be  significantly  larger  than  the  median  of  the  distribution  is  the 
occa-ionai  occurrence  of  dependent  sensor  information.  For  example,  if  most  of  the 
sensed  point  -  happen  to  he  on  a  single  face,  the  amount  of  independent  information 
about  trie  o'ptctT  position  is  much  smaller  than  when  the  same  number  of  sensed 
points  he  on  different  fares.  While  the  sensing  strategy  used  here  will  reduce  the 
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t ho  error  cone  about  the  sensed  normal,  m  mourns,  and  a  bound  on  the  n.agni*  ude 
of  the  position  error,  in  inches. 


One  reason  that  the  maximum  number  of  feasible  interpretations  can  he 
significant  ly  larger  than  the  median  of  the  d:>i :  ibutmn  n  the  e ■  ■  »■  :&>  ion  a!  (■»«,urrenrc 
of  dependent  sensor  information.  For  examp!-  .  ;f  most  of  the  m  v-ed  po;;:*>  happen 
to  lie  on  a  single  lace,  the  amount  ot  mdi  :;w  dent  mlorumtion  .a.i.i  ;'  ti.t  ujict  *  s 
position  is  much  smaller  than  when  the  -..me  number  or  humu  pom!  >  lie  on 
different.  faces.  While  the  sensing  strategv  ;;>*  d  here  will  reduce  ;he  probability 
of  this  occuring,  there  is  still  a  nonzero  cL-.m  e  of  rrdund.mt  ser  >:m;  'nking 

place,  resulting  in  an  occasional  case  of  a  ! .  •  r :  ■  nun  her  of  basil,!*-  inter:  ions. 


The  probability  of  such  redundant  se:m::m  >  ;i!v)  to  a  reran,  extern  dependent 
on  the  shape  of  the  object.  For  example,  net*  ’hm  ’be  aspre:  ratio  uf  t},<  stapler 
is  much  longer  than  that  of  the  motor  hnusu  !  h;s  v.uYld  tend  to  suggest  that 
a  regular  sensing  strategy  is  more  iikely  *o  void  redundant  information  for  the 


stapler  than  the  housing.  Ii 

’.deed .  a  comparison 

,  y  ,  i  r.  . 

uppropriat  e  sect  ion 

s  of  Table 

II  shows  that  umbr  similar 

corn!  it  urns  in.  r ; . » 

ren.en* 

e: ror.  the  number  i 

•  f  feasible 

interpretations:  of  the  stapl 

er  is  much  higher  * : 

c  •  i  ’ ; .  (. 

motor  housing,  cvi 

•n  though 

the  stapler  has  fewer  faces. 

This  IS  partly  due 

o  rrd  ; r 

id  ant  sensing  and  a 

Iso  partly 

due  to  symmetric  interpretations  of  the  data. 


The  number  of  distinct  transformations  :s  almost  always  l  in  these  simulations. 
It  was  also  observed  that  the  computed  transformation  was  generally  very  close  to 
the  actual  one.  For  example,  each  row  of  Table  Y  illustrates  the  average  error  in  the 
computed  transformations  over  100  runs  of  the  algorithm.  The  direction  column 
lists  the  average  angle  between  the  correct  and  the  computed  direction  of  rotation, 
the  angle  column  lists  the  average  angle  between  the  correct  and  the  computed 
magnitude  of  rotation  about  the  rotation  direction,  and  the  translation  column 
lists  the  average  magnitude  of  the  difference  between  the  correct  and  the  computed 
translation  component  of  the  transformation.  It  can  be  seen  from  the  table  that 
the  average  error  is  remarkably  low,  generally  on  the  order  of  2  3  degrees,  even  for 
different  objects  and  different  amounts  of  sensor  error.  As  might  be  expected,  the 
average  error  does  tend  to  rise  with  increases  in  the  sensor  error.  In  no  case  did 
the  algorithm  discard  the  correct  interpretation.  Note  that  the  errors  illustrated  in 
Table  V  wore  recorded  from  the  difference  between  the  correct  transformation  and 
the  computed  transformation  for  the  correct  interpretation.  There  will  be  other, 
erroneous  interpretations,  with  much  larger  differences  between  the  computed  and 
correct  transformation. 
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counted  as  being  distinct.  '[in*  second.  mom  :t 
sy  mmctxic  in’erpret  at  ions  of  the  dal  a.  tor  e>;a 
relative  to  the  sensor,  (onsuier  lirst  the  oi't  w 
a s  tf;e  simple  hand.  \vh:ch  has  *•;  ro‘a toad  y. 
always  found  at  least  two  (list  met  transfo  T:ia’  i 
with  the  sensed  data.  I 'or  objects  such  as  tiie  i 
are  symmetric,  for  example,  the  base  of  the 
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the  measurements  yields  two  nearly  identical  (j.e.  d  me  ring  by  only 
rotation)  transformations  that  account  for  the  data.  As  tin.  <  rror 
data  decreases,  these  multiple  interpretations  tern:  'o  disappear. 
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The  simulation  data  listed  in  Table  IV  is  derived  fror 
interpretations  based  strict iy  on  the  rotation  component  o 
‘hat  is.  two  *  ransfo:  mat  ions  whose  direction  of  rotation  uifa 
were  considered  to  bo  par?  of  the  same  cluster.  This  cii.ste 
very  tight  :ri  the  lotatioa  component,  ignores  possible  < i i HV ; ( i 
'omponent  of’  Mm  transformation.  To  examine  such  diivr«’ii 
simulations  weie  ruin  using  a  clustering  of  the  i.unp  w..: 
sensit i v it y  of  1..Y’  and  a  translation  sensitivity  of  uHuu  0.0:- 
of  distinct  ? rajisformat  ions  under  this  clustering  scheme  are  i 
Note  that  while  the  number  of  distinct  transformations  dot 
he  corM’spoiiding  entries  in  'fable  IV,  the  mango  is  not  signi 


n  a  clustering  of  the 
\  Hie  transformation, 
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>c< *.  in  ? h<  translation 
e*-s,  a  number  of  the 
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admaPui  m  Table  VI. 
s  increase  relative  to 
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•1.4.  Random  Sensing 

All  of  the  previous  simulations  have  generat  ed  the  sensed  data  In  projtr’mg 
a  regular  grid  of  points  along  three  orthogonal  directions,  generally  m 

between  12  and  20  contact  points.  Such  a  sensing  strategy  would  be  ronsi>v:it  with 
visual  sensing  modalities.  A  second  set  of  simulations  has  been  run  using  a  ^  nsmg 
strategy  more  consistent  with  tactile  sensors.  Consider  a  set  of  three  mutually 
orthogonal,  directed  rays,  which  intersect  at  a  point.  Suppose  this  point  is  taken 
to  be  some  arbitrary  point  (x,y,0),  chosen  on  the  j  —  y  piano  (note  that  by  the 
definition  of  the  object  models,  this  plane  will  interoct  the  object).  Mach  ray  is 
traced  along  is  preferred  direction,  (with  decreasing  z  component),  until  either  the 
object  or  the  support  plane  was  contacted.  This  operation  was  repealed  for  several 
different  approaches,  using  randomly  generated  values  of  i  and  y,  until  between  7 
and  9  different  contact  points  were  made  on  the  object.  Tables  VII,  Y1I1  and  IX 
summarize  the  results  of  running  sets  of  simulations,  using  sensory  data  generated 
in  this  fashion. 


Table  VII  -  No.  of  Interpretations  After  Local  Pruning 
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the  following  situation.  With  the  exception  wf  one  pro  t v»  .nt:  pm-i;on,  (sec  Figure 
6).  the  housing  is  essentially  a  symmetric  object,  will:  resix  ct  'o  two  different  axes. 
As  a  consequence,  if  the  sampled  data  points  do  not  he  or.  tins  disi inguishing 
projection,  there  could  he  several  consistent,  sy  mnn  :  ;:c.  inter;  ret  a;  ions  of  the  data- 
in  tne  case  of  sensory  sampling  on  a  regular  grid  of  ponts.  i;  is  likely  that  at  least 
one  point  will  lie  on  this  projection,  and  the  symmetric  ambiguity  wii!  not  arise. 
In  the  case  of  fewer  sample  points,  general  d  by  random  approaches  to  the  object, 
it  is  much  more  likely  that  the  feasible  transformations  will  re-b’et  this  symmetry, 


and  thus  be  higher  in  number. 

In  cases  of  ambiguity  in  interpretation,  for  exam,  when  several  orientations 
<if  the  motor  housing  are  consistent  with  the  s<  used  data,  due  4o  a  partial  symmetry 
of  the  object,  it  would  be  useful  to  have  effective  menus  for  list ingmdimg  between 
the  possible  solutions.  A  straightforward  method  ,vo;;h:  hr  n,  ;:im  M-::sory  points 
generated  at  random  until  only  one  interpretation  is  coi.sw:  t .  'I  In;*,  of  course, 
could  be  very  irieHicient,  since  it.  could  take  the  nddkm:;  »  !  -  v;  points  before 
a  solution  is  found.  In  the  case  of  the  motor  hom-mg,  tor  c;-;n;nrdr.  one  would 
nerd  to  consider  additional  sensory  points  uniii  mm  iv:mt  on  ;! a  piojectirig  lip 


of  the  housing  is  recorded.  A  more  effective  solvi-ou  n  to  us*'  ti  e  difference  in 
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feasible  interpretations  to  find  directions  along  which  the  points  of  contact  of  the 
different  interpretations  are  widely  separated.  Such  directions  then  constitute  good 
candidates  for  generating  the  next  sensed  point  [(iaston  and  l.o/ano- Perez  83 j . 
Kxtcnsions  of  the  method  to  the  six  degree  of  freedom  problem  are  currently  under 
investigation. 

1.5.  Tree  Pruning 


Tables  X  and  XI  contain  a  final  set  of  statistics  that  demonstrates  the 
effectiveness  of  t he  local  contraints  in  reducing  the  number  of  feasible  interpretations 
in  the  IT.  The  regular  grid  approach  is  used  to  generate  the  sensory  data.  J  o r  the 
data  in  'Table  X.  the  points  are  sampled  in  random,  order  as  tin  IT  is  generated 
and  pruned.  For  the  data  in  Table  XT  the  sensed  points  are  sorted  on  the  basis  of 
pairwise  separation,  with  the  more  distant  points  being  ordered  first.  This  soiling 
[  on  distance  tends  to  place  the  most  effective  constraints  at  the  beginning  of  the 

\  process.  Since  the  point  of  the  local  constraints  is  to  prune  the  IT  as  efficiently  as 

possible,  applying  the  most  effective  constraints  first  should  result  in  pruning  out 
i  entire  subtrees  at  as  early  a  stage  in  the  tree  generation  process  as  possible.  Using 

'  the  sorted  sense  data,  the  interpretation  tree  was  generated  and  pruned.  Tables  X 

and  XI  list  statistics  of  the  number  of  interpretations  at  each  level  of  the  tree,  (i.e. 
the  number  of  ^-interpretations  for  different  values  of  k)>  based  on  trials  of  100 
simulations  each. 


Table  X 
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It  can  he  seen  that  the  median  number  of  Tas.bn  *!/•.  r;n’''‘  .*8  ions  is  quite  small 
at  all  levels  of  the  tree,  even  as  the  number  or  contact  points  n-  . : .  c '  mtsim .  1  his  data 
implies  that  one  of  the  strengths  of  the  approach  is  r  -  abm-y  to  prune  out  whole 
subtrees  of  the  IT  at  a  very  early  stage,  thereby  ensuring  Mint  the  Lota!  number 
of  tests  to  be  applied  is  significantly  Mealier  tun::  :m  <2  *  at  entire  iree.  This 
leads  to  very  efficient  processing  of  the  feasible  ;,;UTpr  :  at  r.;:«s 

Sorting  the  points  on  distance  is  extremely  uieci.w:  as  can  be  seen  from  the 
results  reported  in  Table  Xi  of  the  same  set  of  runs  as  u\cse  in  Table  X,  but  where 
the  points  were  sorted  prior  to  pruning.  The  effect  on  running  times  of  the  pruning 
program  is  also  quite  drastic. 

5.  Performance  on  Range  Data 

We  have  performed  limited  testing  of  the  algorithms  described  above  using 
high  quality  range  data  obtained  from  a  laser  nn-ed  irum  mku ion  system  developed 
by  Philippe  Brou  at  our  laboratory.  Two  samples  of  the  data  we  used  are  shown  in 
Figure  9.  The  data  is  obtained  at  high  resolution,  approximately  0.04  centimeter 
grid  spacing  along  j  and  0.08  centimeter  along  y .  A  small  number  cT  points  were 
obtained  from  the  dense  data  by  choosing  points  where  a  hast  squares  fit  to  a 
plane  over  a  5  \  5  patch  produced  very  low  normalized  residue  errors.  Points  were 
chosen  that  included  at,  least  three  independent,  normals.  Note  that  the  actual 
<l)jeel  includes  a  protrusion  that  was  not  present  in  the  model;  no  data  was  taken 
from  that  region.  In  the  data  from  figure  9(a),  eleven  points  were  used;  in  the  data 
from  figure  9(b)  rime  points  were  used.  T  he  accuracy  bounds  we  employed  were 
inch  position  accuracy  and  dr  \$  accuracy  in  measuring  the  normal. 

F;gure  9  shows  the  results  obtained  from  running  tin  algorithm  oi;  the  data 
described  above.  There  were  only  9  and  11  interpret ,m ions,  respectively,  left  in  the 
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tree  after  pruning  with  the  local  constraints.  From  these,  three  valid  transformations 
were  found  in  one  case  and  two  in  the  other;  they  are  shown  in  the  figure.  The 
correct  transformation  was  found  each  time.  The  other  transformations  correspond 
to  rotations  that  place  the  sensed  points  on  parallel  faces.  Note,  however,  that 
disambiguations  between  the  valid  transformations  would  be  straightforward  once 
the  transformations  are  known. 

The  quality  of  the  data  used  in  the  experiments  illustrated  in  Figure  9 
corresponds  to  nearly  the  best  error  conditions  used  in  the  simulations.  Results 
with  larger  error  bounds,  using  data  from  sections  where  the  data  is  less  accurate, 
showed  results  similar  to  those  in  the  simulations,  i.c.,  more  legal  interpretations 
in  the  tree  and  more  valid  transformations  but  always  including  the  correct  one.  It 
tends  to  reinforce  the  validity  of  the  conclusions  found  in  the  simulations. 

6.  The  Combinatorics  of  Pruning  the  IT 

In  the  previous  sections,  wo  have  outlined  the  basic  interpretation  algorithm. 
The  crucial  issue  that  determines  the  viability  of  this  algorithm  is  the  effectiveness 
of  pruning  the  interpretation  tree.  Our  goal  has  been  to  demonstrate  that  one  can 
use  simple  local  constraints  to  prune  the  interpretation  tree,  so  that  only  a  few 
of  the  relatively  expensive  model  tests  need  to  be  made.  The  simulation  results, 
under  a  variety  of  conditions,  and  the  results  on  range  data  provide  support  for 
this  claim. 

It  is  also  possible  to  provide  a  combinatorial  analysis  of  the  pruning  of 
interpretation  trees  provided  by  local  constraints.  A  detailed  presentation  of  such 
an  analysis  is  contained  in  a  companion  paper  [(crimson  and  Lozano-Perez  83]. 
Here,  we  demonstrate  the  scope  of  the  combinatorical  analysis  bv  presenting  a 
detailed  discussion  of  the  use  of  the  distance  constraint  in  pruning  interpretation 
trees.  Similar  results  hold  for  the  other  constraints. 

We  stress  that  the  results  given  below  are  actually  weak  bounds  on  the  number 
of  interpretations  to  be  expected  after  pruning.  In  practice,  numbers  close  to  these 
bounds  are  observed  only  when  the  sensors  are  arranged  so  as  to  obtain  a  minimum 
of  information  about  the  object. 

6.1.  Combinatorics  of  Distance  Pruning 

We  will  consider  the  case  in  which  all  faces  (or  edges  in  the  two-dimensional 
case)  have  the  same  size,  and  derive  bounds  on  the  expected  pruning  of  the  IT. 

Assume  we  have  some  arbitrary  labelling  of  the  faces  from  1  to  n  (for  example, 
in  the  two-dimensional  case,  based  on  arc  length  from  some  starting  point).  For 
each  pair  of  faces,  i  and  j,  let  dtJ  denote  the  separation  of  the  midpoints  of  the 
faces.  Let  ci;  be  an  upper  bound  on  the  range  of  variation  in  distance,  for  different 
sensed  points  on  the  two  faces,  i.e.  let 

et;  =  lim  sup  je:  —  e  <  jx  —  y  |  <  dtJ  -f  r,Vx  on  face  i,  Vy  on  face  jl 
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where  j\  -  y|  is  the  distance  between  point  x  on  face  i  and  point  y  on  face  j.  Let  t 
be  defined  as  the  maximum  over  all  Lj  of  plus  some  estimate  of  the  maximum 
error  of  the  sensed  distance. 

Now  assume  that  we  haw  recorded  '!.»■  posi  :on  of  two  sensor  points.  and 
/j.  and  let  be  the  measure:  distamv  helwitn  the;n.  Assume-  th;il  the  tir-i  point 
lias  been  arbitrarily  assigned  fo  some  fare  ?  m  the  object.  \\e  want  to  determine 
how  many  faces  j  of  the  object  can  rumi-Jem  ;y  be  m.-igaed  to  the*  second  point 
given  the  separation  s i -j  and  the  known  ow  ninr  ;<>n  of  dim ances.  M.iiic'er,  wo 
want,  to  be  able  to  continue  this  for  A  -«:r<u  poiu'.s.  drirrmimug  an  upne;  bound 
on  the  number  of  assignments  of  faces  to  -emm  point-  * ii a*  are  consist <  m  with:  :he 
sens  mi  separation  between  the  faces. 

Let  the  distribution  of  faces  with  re-pec’,  ‘o  mw  ;  ;s  a.  fum  t:o:i  of  d -stance  be. 
cenoo  t:  by  />.(>■.  In  oilier  words,  pp.-'  rei-ord--  ■).<  n.mmrr  «  :  ac<  •  .vi  -no  m.d point, 
separators  from  !ace  ?  is  given  by  h:e  dmomu  As  ,-i  *  om-epuenee. 

/  ///di)  '  u 

-L  o  v 

where  n  is  the  total  number  of  faces,  ami  d  is  trie  diameter,  or  m axuuum  reparation 
of  th.e  object.  Note  that  because  dp.  n  a  dis:  ubut  ion.  this  is  1  .rbrsgue-M  iril  jes 
mtcg;a!.  ‘Hie  following  bound  on  the  number  of  nodes  at  the  klr  live!  or  the  IT 
holds  for  both  two-dimensional  and  \  iin'e-.iinietmumal  objects. 

Proposition  1;  An  upper  bound  on  the  expected  number  of  nodes  at  the  kth 
level  of  the  interpretation  tree,  k  >  2,  is  given  by 

2  ni\^  1 

n 

d  ) 

where  d  is  the  diameter  of  thr  object,  and  c  is  a  bound  on  the  distance  sensitivity 
of  the  model. 


Proof:  The  proof  proceeds  by  considering  an  iterative  application  of  the 
expected  maximum  branching  factor  at  each  level  of  the  tree.  We  assume  that 
bk.  i  denotes  a  bound  on  the  number  of  consistent  nodes  at  the  k  -  Lsf  level  of  the 
interpretation  tree,  and  consider  the  brandling  factor  obtained  when  adding  a  k*h 
sensed  point.  Assume  that  sensor  point  P k  \  has  been  assigned  to  face  ?,  and  that 
the  measured  separation  of  sensor  point  I\  i  and  /\  is  >>.  This  implies  that  the 
midpoint  separation  of  the  corresponding  faces  is  within  (  of  sk.  Hence,  an  upper 
bound  on  the  number  of  possible  faces  consistent  with  sk}  given  face  i  assigned  to 
point  is 

/  dpx{sk  -f  x). 

Since  the  number  of  nodes  at  the  k  --  Ist  level  of  the  tree  is  bounded  by  bk-u  an 
upper  bound  on  the  total  number  of  nodes  at  the  kth  level  of  the  tree  is 


bk  j  max  /  dpt{sk  +  i). 

t  Jz  c 
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t;.( Toby  concluding  the  proof  by  induction.  | 


While  tiiis  proposition  giv«*s  us  an  iipper  <a  *:a  ..  xpu  i  *o(j  number  of 

r  ( ■  d  o v’ .  m  order  *o  evaluate  it  \v<  neeo  some  m.,  m  on  t  \  lie  lo!i(»\vmg  two 
propositions  provide  this  for  the  two*  and  three- a-  ,e:  c.isos 

Proposition  2:  If  all  the  edges  of  a  two-dimensional  .  hject  have  the  same  length 

r,  then  Ji,  j  <x)  <  t. 

Proof:  Connect  the  midpoints  of  two  arbitrary  fares,  i  amt  /.  with  a  line  of 
length  dtj  (’onsider  first  the  case  of  (il}  t  t\  Tor  >oi  {y  j\\  po--ible  uurnt.at  ions 


(inrr.ion  A  Lotaro-FVrpi 


‘<r: 


figure  10.  Illustration  for  Proof  of  Proposition  2 


o:  carl:  T  t :m  edgr<  about  its  midpoint  describes  a  circle  of  rad.-is  >  abo  ;T  ■  h.y 
e:u:po::.t.  We  aro  intcrey «  d  n:  the  f-xtrema  ;a  sopum’  h-:.  of  :kom>  m  *..r>v  uiss.- 
.see  }-  :e;;i re  10:.  WV  chum  mat  the  maximum  and  minimur:;  Mparavon  of  points  in 
me  uisks  occur  tor  t  :.t  ease  of  me  edges  parahei  to  tre  ru-cno.nt  r<  nneetc  \ 
a  minimum  of  d<  ?  and  a  maximum  of  dJ7  -re. 

While  this  can  be  shown  algebraically,  there  is  also  a  simple  geometric  ;  roof. 
Construct  a  coordinate  system  with  origin  at  the  midpoint  of  edge  i  and  v.  r:(  r  axis 
along  the  midpoint  connector.  Now  construct  a  circle  o!  radius  m  -  t  d-  ■  ;*  :  i : e 
point  ;<v  —  W  Of.  C  learly,  this  circle  grazes  the  first  disk  at  the  pom;  ,  n  .  Now.  m 
order  for  any  other  point  in  the  second  disk  to  have  a  shorter  distance,  we  must  be 
able  to  position  a  circle  of  the  same  radius  about  that  pom*  and  still  intersect  the 
first  disk.  This  is  not  possible,  by  the  following  argument  The  envelope  of  possibn 
points  can  be  formed  by  sweeping  a  circle  of  radius  dr.  —  t  through  a  series  of 
positions  such  that  the  center  of  the  circle  lies  at  the  limit  of  tne  second  disk  This 
envelope  only  intersects  the  first  disk  at  the  above  mentioned  point,  and  hence,  the 
mini  mu  mi  possible  separation  between  the  two  edges  is  given  by  dx,  —  e. 

Similarly,  the  maximum  separation  can  be  shown  to  be  dx  ~~-e  by  constructing 
a  circle  of  radius  d,,  —  c  about  the  point  (dt;  ~r  NO)  and  using  the  same  argument. 

If  dt;  <  e,  then  the  minimum  distance  is  clearly  bounded  below  by  0.  The 
construction  for  the  bound  on  the  maximum  distance  is  identical  to  that  a)  ;e. 
Hence,  we  see  that  etJ  <  | 

Corollary:  If  all  the  edges  of  a  two-dimensional  object  have  the  same  length, 
and  the  sensor  error  in  measuring  distances  is  much  smaller  than  the  length  of  an 
edge,  then  the  expected  number  of  nodes  at  the  ktfi  level  of  the  interpretation  tree, 
k  >  2,  that  survive  distance  pruning  is  bounded  above  by 


where  p  is  the  perimeter  of  the  object,  and  d  is  its  diameter. 

Proof:  S  nice  the  sensor  error  is  much  less  than  the  edge  length,  we  see  that  (  is 
essentially  given  by  the  maximum  over  all  i,j  of  (tJ.  From  the  previous  proposition, 
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' !  ,\  . . .  :  J  T * : .  i  '■  oT!  i.’> l m  0 .  < 1  *  >  "  ' .  -  •  ’  .  ! ! .  *  *  r  •.  r  , .  '  *  , •. >■  •  ,/(’('*. 

Pro,  ."it  ion  a:  If  nil  ihc  lari'-  of  a  • h  r e;  - « !  m  i  inio!.;! !  o  i  < ;  a  t  *  •  haw  the  mine 
diameter  <  .  and  the  same  area  \  *  then  '  '  n 

Proof:  !  ,dr:/v.d  Her.  ‘he 


(  orollar>  :  [fail  flu  hues  of  a  three-dimensional  »du*  \  i  i  ^  <•  <  h  <  same  diameter 
ami  ihe  same  surface  area,  and  the  >rn>or  em>r  in  measuring  distances  is  much 
smaller  lhan  lln  diameter  of  a  fare.  fl.t  fi  flic  v -men  d  mum  n  .mdrs  at  the  k:** 
le\e!  of  thr  inter-relation  tree,  k  *  ;j:at  Mir\m  dh.aurr  »»nuim£  is  hounded 
above  by 


where  \  is  the  surfarr  area  of  the  ohjeri  and  u>  dnuiirfer. 


Proof:  :  h  r  s#  d:.u  anee  ^  nund;  less  than  the 

u1  o:  a  v.*  *  ‘••a*  r  >  m -u:  m l.\  ruw.  u.  ‘:.i  :;mxi  mum  over  aM 

’  ..  /  :  j':,  *  ,v  pnvimj-  nroposnon.  'h;-  >  menaced  i»y  ;i;e  I  a  r  t  uiarnctcr  c.  If 

.  \  rs  t  lit  si:r  area  of  ’hr  fare,  t.iirr*  \  .  '•  H  p.  Vu\  rover,  A  f  -----  so  that 

'  •  '  •  ’  2\  u'.:  ’.or  enrol,  ary  follows.  | 


if 


>rn 


r’  :s  ri;::, (■;•:,  then  -he  area  A  is  boninied  .move  in  rr  ri " ,  and  the 
i hu  es  to 
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it: : t  :  pri  tat  urns  to  be  further  reduced. 

1*  was  a  surprise  to  the  authors  that  weak  upper  hounds  cm  the  number  of 
interpretations  would  he  less  than  exponential  in  the  number  of  sensed  points,  k  (for 
example  in  the  three  decree  of  freedom  ease,  where  the  number  of  imerpn. tuitions 
is  linear  m  the  number  of  sensed  points).  In  our  experience,  however,  many  people 
find  it  surprising  that  any  of  the  bounds  should  grow  with  k  Most  people  expect 
them  to  deennse  with  k.  i.e..  as  more  points  are  acquired,  the  constraint  should  be 
tighter.  Recall,  however,  that  the  bounds  derived  above  do  not  take  into  account 
tfie  fact  that  there  are  (*)  distance  constraints  at  the  kth  level  of  the  tree;  they 
only  apply  a  single  constraint  at  each  level  of  tin*  tree.  There  is  mother  important 
effect  that  (partially)  accounts  for  the  growth  in  the  number  of  interpretations 
with  k.  Namely,  that  for  k  <  (>  each  interpretation  corresponds  to  a  continuous 
range  of  positions  and  orientations.  For  example,  for  k  --  1.  each  interpretation 
corresponds  to  the  whole  space  of  positions  and  orientations.  As  more  points  are 
added,  the  volume''  in  the  space  of  positions  and  orientations  consistent  with  each 
interpretation  decreases,  bn*  the  number  of  these  interpretations  may  increase  (as 
they  do  between  k  1  and  k  —  2)2. 


7.  Discussion 

It  is  important  to  note  that  the  algorithm  described  in  this  paper  has  quite 
low  computational  cost.  The  pruning  algorithm  is  particularly  clhcient.  The  range 
tables  store  all  the  model  information  needed  and  pruning  is  done  by  simply 

"Wo  are  indebted  to  John  Canny  for  this  observation. 
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comparing  tin1  ranges  of  v.bues  : 1 1 < * . pi  :>  >•:  a.../,  ■  '  -t;:m  ' -  - ;  w(.; 

in  the  tables.  I  herefore,  rio  ari?hm<  /  as  done  a’  .rmg  ;  Vm  •  .-■*  ror  nan-xing 

into  tables).  It  is  uni)  l!ie  model  test  trial  requires  .my  sg  mm  ,.M  i  vm.p  .q  ation  and, 
therefore,  the  desire  to  minimize  tin  numtu  :  m'  ‘  ,:;:o  V,  a.-.,-  .»*.  .ePomned. 
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expanded  on  (';e:.  ..  Or  a;r-  k  was  </.L’  I  *«.  ••ora!'.  In  n-  *  ;  age  tm  e  to 

coinpie:  ••  4  e ;  <  m ;  •  . ; ,  <  . !  r ,  a  a-  ‘ : : .« a  ;■  m.  •  or, o *• ,  1 . >i  t  •  .  n  .r  nu'i.f  .*.!  :ou . 

alt  Hough  th>  :  . r.  :  won  at  ore. monad v  nc  r:.a  «  o»  <:  :n  '  »  tons  i n  a  Mi  !; 

a  iargx  numm  r  w  rrpie’. a?  ions  were  possible. 

i  m  1m'...  ’ .  * ’  . . .’  n  •  ■  t  n o :  1  w  v»»>  .m  *  i  r . :  n  I  '  n *  n ..e i  ,*  (tat  a, 

be  dr  on  n  fr  >m  m  a'  I  h":  ..*  n: in  o-‘  ma; 

com.::;;,  w her.  1 1 . .  .)!.,» •  !  a  ;i  n;;  n  .i;,y  n  o.^v  *  /'  e  .  .  n  .  nyp  si  s 
is  made  than  ad  tie*  points  be  iota,  to  om  (/?.,«-  :  m,  ;.tl:ni;r-  m  *  rpret  m  ions 

are  found,  Wan  one  i  am  teii  that  the  hypothesis  m  v,  Much,  more  research  is 

needed  ;n  ’bis  area,  however. 

I  drougho  a  t  the  paper  we  nave  umri-u  »>ui  m :  <  mmo:  to  tie  iiuii.tnr  ot 
iuterpre: . ations.  relative  to  one  model,  of  nnia  obtained  mom  that  obu-  i'o 
carry  tail  recognition  between  severed  objects,  om-  determines  ‘in  ..umber  ot  .ega, 
interpretations  of  our  set  of  data  relative  to  mmtqm  object  models  I'his  \  roces> 
can  simply  ne  performed  -equeir tally  on  each*  mooei.  One  simple  improvement  is 
dearly  possible.  If  one  stores  vmth  each  r  eJ- •.  da  mum  balance  between  any 
of  the  faces,  tlien  if  on<  of  Uu  measure;:  instances  is  greater  hnnu  tins  jpper  hound, 
the  model  can  bo  discarded  at  v  aa  h  i*is  technique  on. only  separates  iarge  objects 
from  small  ones.  1  fo; ;  mmt e!y,  very  s:  ml;  m'.emoh  d e-t  ,-m  !>•  not  rule  out 

large  objec  ts.  A  second  method  would  be  lo  use  ii i re*  t  ion  histograms  to  rme  out 
certain  models,  lorn  example,  if  the  angle  biiwren  two  miim,;  imruuds  was  30V 
then  a  model  of  a  cube  would  not.  be  consistent  \v;t!i  this  a.  and  could  (juukly 
hr  excluded. 

After  mmeralinp,  and  pruning  t.he  intn  [)retaliou  tr^  e  a:iu  p*  rformine  t hi  model 
test  on  each  of  the  k*  own  objects,  \vc  have  a  listing  oi  nil  the  positions  and 
orientations  of  all  objects  consistent  with  th  *  measured  data.  At  this  point,  flirt  her 
discrimination  can  be  carried  out  by  additional  unguided  sensing  as  before  or 
by  consul  ’";ng  the  ai'ernativcs  and  choosing  a  good  place  to  musc  next.  The 
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recog  nit  ion  problem  tha’  remains  is  now  amenable  to  n*her  techn Kjiics  a>  c.rd  since 
it  has  brer,  minced  to  the  mueh  more  trartabie  problem  of  <: JTeri  m  a'  an:  among 
a  class  of  objects  :n  known  pu>;t>on>  and  orient  a!  ions. 

Acknowledgements 

Philippe  Hron  contributed  freely  of  \w<  tune  and  efoit  to  pro\i<ie  n-  with  *  he 
range  data  used  in  our  experiments;  we  are  very  gratCa;.  \\c  uN.<  mm:*  mnw-d  his 
comineiits  on  the  presentation  o*’  this  paper.  We  arc  <•.  than--.:  h  *<.  h  R.  hc< 
and  Bert  hold  Horn  for  their  dot  abaci  comments  and  >nn  a :o;>  on  ,n.  t  >.  a  :  baft 
We  have  also  henelited  from  discussions  with  Olivier  Faugeras. 

References 

Agin,  G.  J.  and  T.  0.  Hinford  ‘Computer  D«  <e”ip'  ion  of  Cnr.vd  Oh:<vv/‘ 
Third  Intcrnatio t:dl  Joint  (  i>nft  r(  net.  <  n  AriJ'c:.;!  ;\;C.  •(/.*  \  •.  .  p)7 \  ml'  1  tip: 

Hajcsy.  It.  “Computer  Identification  of  Visual  Surface,"  <  b-jiib  -  (R  :p hu  t 
and  Image  Proa  ><ing  2.  2  (1973).  1  IS  130. 

Hajcsy.  It.  and  I  .  Licbcrmann  “Ttxture  Gradient  as  a  Depth  <’  a  .  '  ’  m;  mC- 

Graphic s  arid  I >•  age  /  Vo.“<  >"ing  a.  1  (197b),  52  67. 

Bakrr,  II.  II.  and  T.  0.  Hinford  “Depth  from  Edge  and  Intensity  H.eo*d  >*erco,” 
Seventh  International  Joint  Conft  rence.  nri  Artijicial  letdhg*  fir*  ,  A':;.  !0  19S!, 

631  636. 

Hinford,  T.O.  “Sensor  Systems  for  Manipulation,’  lir.moi*'!],  \UiTi*:ctl  Sy "terns 
Conference  ,  1972. 

Holies,  It.  C.  and  It.  A.  Cain  c  ‘Recognizing  and  Locating  Partially  Visible 
Objects:  The  Local-Fcature-Focus  Method,”  Robotics  Research  1.  3  (1982),  57  82. 

Holies,  R.  C.,  P.  Iloraud  and  M.  J.  Hannah  3DPO:  A  Three  Dimensional  Part 
Orientation  System,”  First  International  Symposium  of  Robotics  Research  , 
Bretton  Woods,  N.IL,  August,  1983. 

Rriot,  M.  “The  Utilization  of  an  ‘Artificial  Skin’  Sensor  for  the  Identification 
of  Solid  Objects,”  Ninth  ISIR  ,  Washington,  D.  Ch,  March  1979,  529  548. 

Hriot,  M.,  M.  Itcnaud,  and  Z.  Stojilkovic  “An  Approach  to  Spatial  Pattern 
Itecognition  of  Solid  Objects,”  IEEE  ' Transactions  on  Systems,  Man ,  and 
Cybernetics  SMC-8,  9  (September  1978),  690-694. 

Brou,  P.  Finding  the  Orientation  of  the  Objects  in  Vector  Maps,  Ph.  D.  Thesis, 
Department  of  Elec.  Engr.  and  Comp.  Sci.,  Massachusetts  Institute  of  Technology, 
August  1983. 

Buchanan,  H,,  C.  Sutherland,  and  E.  A.  Fcigcnbaum  “Heuristic  DENDRAL:  A 
Program  for  Generating  Explanatory  Hypotheses  in  Organic  Chemistry,”  Machine 
Intelligence  4,  (E  Melzer  and  I).  Mxchic,  cds)  (1969). 


41 


r 


Unrnion  &  I.u»4no'lVre« 


Mu  del  iiiticd  Hc.n  gmltoti 


Dixon,  J.  k.,  S.  Salazar,  and  J.  It.  Slavic  '  RcM  .tr  .  >:  LicSie  S<-ns ors  for  an 
Intelligent  Robot,"  Xinth  ISIR  ,  Washington  i).  ('.,  March  197th  507  518. 

Kaugeras,  ().  1).,  v\  al.  "Toward  a  Flexible  Vision  System/’  Robot  Vr,:on,  A. 
Pugh  [editor),  IKS  Publications,  United  kingdom,  !983. 

Faugeras,  O.  I).  and  \1.  Hebert  ‘  \  3-R  IT -cognition  a/ni  I *<>' :? i < > : i : ; i r.  Algorithm 
using  (leometricai  Matching  between  Primitive  Surfaces,"  TTyhiri  Inti  r national 
Joint  Conjtrvnct  on  Artijical  Intcllujt  net  .  1983,  996-1002. 

(iaMon,  l\  C.  and  1.  Lo/ano-Pcrr/  '“Rutile  Recognition  and  Localization 

1  sing  Object  Models.  Mi  l  Artificial  Intelligence  Laboratory.  AIM  705,  1985. 

(irirusou,  \\  .  IT  L,  A  Computer  Impaum  m  m  nu.  of  ;i  'Timy  « . r’  IF. mar;  Sfereo 
Visum."  / ,h.i!o>oj>hictil  /ninMirCom-  <>]  *v  R  n,a.  Arm.'  A  Tendon  li  292 
[DM),  217-253. 

Crimson,  W.  IT  I  .  From  hnagt  -  9  Sn' r  .-  .  \  ,».►  :  e m  Andy  o/  Hie 

human  tarty  vision  system  .  MI  1  Pic^.  t  mne/nn  1981. 

C  •  rim  son ,  W.  I  .  L.  A  ( 'omput  at  joua!  i  F  o:  \  of  \  n:;i.  s-.-fner  Interpolation," 
Phil  l  n:n>  R(-y.  A-  Ton<i  R  29^  iOMT.  D5  127 

Crimson,  U.  IT  L.  Ari  !;:ipidn<  *  .  <  •  .pvm  mrai  theory  of  Visual 

Surface  Interpolation, "  Cornpnftr  Wsmn.  i  P\ipn ana  /rump  / Voce  ssmg  22 
(1983),  39  69. 

Crimson.  W.  li.  L.  and  IT  Lozano- INTc /  "A  t  omAnm  onai  Analyses  of 
Recognition  and  Localization  using  Object  Mourns,”  Alii  Artificial  Intelligence 
Laboratory,  to  appear,  1983. 

Harmon,  L.  D  ‘‘Automated  Tactile  Sensing, 1  Robotics  Research  l,  2  (Summer 
1982),  3  32. 

Harmon,  L.  I).  “Touch-Sensing  Technology :  A  review,”  Society  of  Manufacturing 
Engineers,  MSU80-03,  1980. 

llillis,  \V.  I).  A  High-Resolution  linage  Touch  Sensor/’  Robotics  Research  1, 

2  (Summer  1982),  33  44. 

Ikeuchi,  K.,  Horn,  B.  K.  IT,  Nagata,  Callahan.  T.  and  ().  Kcingold  “Picking 

Ip  An  Object  from  A  Pile  Of  Objects,  ’  First  International  Symposium  on 
Robotics  Research  ,  Brett  on  Woods,  NTIL,  1983. 

Ikruchi,  K.  and  B.  k.  P.  Horn  “An  Application  of  Photometric  Stereo/’  Sixth 
Inti.  Joint  Conf.  on  Artificial  Inttll:  /<  f.  c  ,  1979,  913  415. 

Ivan c.e vie.  N.  S.  "Stereometric  L-itern  Recognition  by  Artificial  Touch,” 
PatUrn  /TV-  ocpvtion  6  (1974),  77  83. 

Jarvis,  It.  A.  “\  Perspective  on  Range  Finding  Techniques  for  Computer 
Vision/'  IFFF  Trans,  on  Pattern  Analysis  and  Machine  Intelligence  PAMI-5, 

2  (March  1983),  122  139. 

Render.  J.  It.  “Shape  from  Icxtun  Carnegie  Mellon  University  Computer 
Science  Report.,  CMl  CS-8M02,  1980. 


42 


Crimson  &  Loiaho- 


Model  Bail'd  Hr* trillion 


Kinoshita,  G.,  S.  Aida,  and  M.  Mori  “A  i?at torn  Classification  by  Dynamic 
Tactile  Sense  Information  Processing,”  Pattern  Recognition  7  (1975),  213. 

Korn,  C.  A.  and  T.  M.  Korn  Mathematical  Handbook  for  Scientists  and 
Engineers  ,  McGraw-Hill.  New  York,  1968. 

Lewis,  It.  A.  and  A.  R.  Johnston  “A  Scanning  Laser  Range  Finder  for  a  Robotic 
Vehicle,”  Fifth  Inti  Joint  Con/.  on  Artificial  Intelligence  ,  1977,  762  7 68. 

Lozano-Perez,  T.  “Spatial  Planning:  A  Configuration  Spare  \pproa<!\‘  IEEE 
Transactions  on  Computers  C-32,  2  (1988),  108  120. 

Marik,  V.  ‘‘Algorithms  of  the  (Complex  'Tactile  Information  Process, fig," 
Seventh  Inti  Joint  Conf.  on  Artificial  Intelligence  ,  1981,  773  771. 

Mayhcw,  J.F.W.  and  J.P.  Krishy  "Psychophysical  and  ( ’ouipm atn:.H  Studies 
towards  a  Theory  of  Human  StereopsusT  ArtiJi'-:-u  In:>,  ligt  re  17  }9Sp 

Ncvatia,  It.  and  T.  O.  Binford  ‘Description  and  Recognition  of  (‘urved 
Objects,”  Artificial  Intelligence  8  (1977),  77-98. 

Nitzan,  I).,  A.  E.  Brain,  and  It,  O.  Duda  The  Measurement  and  Use  of 
Registered  Reflectance  and  itange  Data  in  Scene  Am ’vslv“  Prji  of  IEEE  65 
(February  1977).  206-220. 

Okada*  T.  and  S.  Tsuchiya  “Object  Recognition  by  Gr  :>pmg.  Pattern 
Recognition  9 ,  3  (1977),  Hi  119. 

Oshima,  Vf.  and  Y.  Shirai  “A  Scene  Description  Method  using  Three- 
Dimensional  Information,”  Pattern  Recognition  11  (1978),  9-17. 

Oshima,  M.  and  Y.  Shirai  “Object  Recognition  Using  Three-dimensional 
Information,”  IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence 
PAMI-5,  4  (July,  1983),  353  361. 

Overton,  K.J.  and  T.  Williams  “Tactile  Sensation  for  Robots,”  Seventh  Inti 
Joint  Conf.  on  Artificial  Intelligence  ,  1981,  791  795, 

Ozaki,  II.,  S.  Waku,  A.  Mohri,  and  M.  Takata  “Pattern  Recognition  of  a 
Grasped  Object  bv  Unit- Vector  Distribution,”  IEEE  Transactions  on  Systems , 
Man,  and  Cybernetics  SMC- 12,  3  (May/June  1982),  315  324. 

Page,  C.  JM  A.  Pugh,  and  W.  B.  Hcginbotham  “Novel  Techniques  for 
Tactile  Sensing  in  a  Three-Dimensional  Environment,”  Sixth  ISIR  ,  University  of 
Nottingham,  March  1976. 

Popplestonc,  R.  J.,  C.  M.  Brown,  A.  P.  Ambler,  and  G.  F.  Crawford  “Forming 
Models  of  Plane  and  Cylinder  Faceted  Bodies  from  Light  Stripes,”  Fourth  Inti 
Joint  Conf.  on  Artificial  Intelligence  ,  Tbilisi,  Georgia,  USSR,  September  1975, 
664-668. 

Purbrick,  J.  A.  “A  Force  Transducer  Employing  Conductive  Silicone  Rubber,” 
First  International  Conference  on  Robot  Vision  and  Sensory  Controls  , 
Stratford-upon-Avon,  United  Kingdom,  April,  1981. 


43 


Crimson  &  LoaanoP£rei 


1 


Hrfd<  d  K»*t  ion 


Uaibert,  M.  II.  and  J.  K.  Tanner  “Design  and  Imp!'  maium  of  a  VLSI  Limit- 
Sensing  Computer,”  Robotics  Research  I,  3  (1982),  3  i8. 

Shirai,  Y.  and  M.  Suwa  “Recognition  of  Pohyhedrons  with  a  Rang*  l'/nder,” 
Second  Inti  Joint  Conf.  on  Artificial  Intelligence  ,  1971. 

Schneitcr,  J.  L.  An  Opt  ical  Tactile  Sensor  for  Robot  s,  S.  M.  1  hosL,  Depart  :in‘!it 
of  Mech.  Lngr.,  Massachusetts  Institute  of  Technology,  August  1982. 

Shneicr,  M.  “A  Compact  Relational  Structure  Representat ion,”  Sixth  Inter¬ 
national  Joint  Conference  on  Artijicial  Intelligence  ,  1979.  818  826. 

Snyder,  Wesley  K.  and  J.  St.  Clair  “Conductive  Klastomers  as  Sensor  for 
Industrial  Parts  Handling  Kquipincnt/*  IEEE  Transactions  on  Instrumentation 
and  Measurement  1M-27 ,  1  (March  1978),  9-1  99. 

Stevens,  K.  A.  ‘‘  The  Information  Content,  of  Texture  Gradients,’*  Biological 
Cybernetics  *12  (1981),  95-105. 

Stojilkovic,  Z.  and  I).  Saletic  Learning  to  Recognize  Patterns  by  Belgrade 
Hand  Prosthesis,  1'iflh  ISIR  ,  l  IT  Research  institute,  Chicago,  1975.  *107  113. 

Sugihara,  K.  “Range-dat  a  Analysis  Guided  by  a  Junction  Dictionary,  Artificial 
Intelligence  12  (1979),  11  69. 

Takeda,  S.  Study  of  Artificial  Tactile  Sensors:  for  Shape  Recognition:  Algorithm 
for  Tactile  Data  Input/'  Fourth  ISIR  ,  197-1,  199  208. 

Terzopoulos,  I).  “Multilevel  Computational  Processes  for  Yi>ual  Surface 
Reconstruction,”  (\ -.impute  r  \’ision,  Graphics  and  Image  Processing  (1933), 

to  appear. 

Wood  ham,  U.  J.  “Photometric  Stereo:  A  Reiloctanee  Map  Technique*  for 
Determining  Surface-  Orientation  from  Image  Intensity,”  Image  Understanding 
Systems  ami  Industrial  Application*.  Fro  FIE  155  ,  1978. 

Wo  odh  a  in,  IL  J.  “Photometric  Method  for  Determining  Surface  Orientation 
from  Multiple  Images,”  Optical  Engineering  19,  1  (1980),  139  144. 

Woodham,  U.  J.  “Analysing  Images  of  Curved  Objects,”  Artificial  Intelligence 
17  (1981),  117  140. 

Appendix  I 

Here,  we  establish  the  claim  of  section  2.1.2  that  the  set 
{ i> i  •  n2  |  ni  •  uj  >  <lf  n2  u2  >  c2} 

is  contained  in  the  set 

{n i  •  n2  |  cos  [min(7r,  0l2  4-  <t>  1  4*  4> 2)]  <  n,  n2  <  cos  [inax(0, 0n  —  <p\  4-  <j>2))} 
where 
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Figure  IF  Extremal  values  of  dot  products  between  two  cones 


COS  <t>\  — - 
COS  (j>2  =  C2 
COS012  =  Mj  *  U2  =  7. 

While  it  is  possible  to  prove  this  algebraically,  it  is  simpler  to  see  this  by  the 
following  geometric  construction  (see  Figure  11).  We  wish  to  determine  the  extremal 
values  of  the  dot  product  between  unit  vectors  in  the  two  cones,  or  equivalently, 
extremal  values  in  the  angle  between  any  two  such  vectors.  If  the  cones  about  u\ 
and  u 2  intersect,  clearly  the  maximum  value  of  the  dot  product  is  1.  If  the  cones 
are  antipodal,  clearly  the  minimum  value  is  — 1. 

We  now  consider  cases  in  which  the  cones  do  not  overlap.  We  claim  that  the 
extremal  values  for  the  dot  product  occur  when  the  two  vectors  lie  in  the  plane 
spanned  by  ui  and  U2,  with  the  vectors  lying  at  the  limits  of  the  cone  within  this 
plane.  That  is,  if  we  let 


then  the  extrema  occur  at 

n,  =  (ei  —  7Pi)ui  -f  P1U2 
n2  =  p2ux  +  («2  —  7P2)»I2 

and 

ni  =  (ti  4-  7Pi)ui  —  P1U2 
n2  =  — P2U1  -f  (C2  +  7P2)»2 

The  first  case  can  be  shown  to  correspond  to  the  minimal  angle  between  vectors 
in  the  two  cones,  by  the  following  construction.  Construct  a  cone  centered  about 
ni  with  radius  such  that  112  lies  on  the  boundary  of  the  cone,  that  is  the  new 
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cone  grazes  the  112  cone  at  112  If  there  is  a  smaller  ;  V,  ::  must  he  possible  to 
reposition  this  cone  so  that  it  is  centered  at  some  othe.  point  in  the  uj  con**  and 
yet  still  intersects  the  U2  cone.  This  is  clearly  not  possible,  and  hence  the  minimum 
value  of  the  dot  product  is  given  by  the  stated  choice  of  iq  am:  iij.  Lxpnndmg  the 
dot  product  for  this  case,  and  making  the  appropriate  t: igommie' ric  substitutions 
yields  the  required  expression.  A  similar  construction  honb  for  the  maximum  angle 
(or  minimum  dot  product). 
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Here,  we  show  how  to  compute  the  range  of  possible  direction  vectors  between 
fact\  and  fcicCj  in  the  object  model.  Let  us  erect  a  coordinate  system  on  facet  at 
the  centroid  of  the  face  and  whose  z  axis  points  along  the  normal  of  the  face.  Then, 
it  is  clear  that  the  set  of  possible  direction  vectors  is  the  set 

{\j  —  v,  I  \j  £  facej  &  vt  £  facet } 

where  both  v:  and  v,  are  expressed  relative  to  the  frame  on  faccx .  Assume,  for 
now,  that  both  faces  are  convex.  It  can  be  shown  Lozano  Perez  83]  that  this  set  is 
equivalent  to 

ch({\j  —  xx  |  £  vert{jac€j)  &  v,  £  vert(facet)}) 

where  ch()  is  the  convex  hull  of  a  set  of  points  and  ?cr^)  is  the  set  of  vertices  of  a 
face.  Because  of  convexity,  the  extrema  of  the  component  of  the  direction  vectors 
along  the  normal  of  facex  occur  at  the  vertices  of  this  convex  hull.  Clearly,  the 
vertices  of  the  convex  hull  of  a  set  of  points  are  drawn  from  the  set  of  points  itself. 
Therefore,  we  need  only  find  the  extrema  of  the  finite  set 

{n,  •  (v;  —  v,)  |  v_,  £  vert(faccj )  A-  v,  £  verf{facet)} 

where  n;  is  the  normal  to  /acet. 

When  the  faces  are  non-convex,  the  procedure  above  will  generate  a  conservative 
bound. 
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